-- Logs begin at Wed 2022-08-31 11:04:34 UTC, end at Wed 2022-08-31 12:09:47 UTC. -- Aug 31 11:28:45.008238 dv-cinder-839768-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 11:28:45.019835 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 11:28:45 2022] *** Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: nodename: dv-cinder-839768-3-iscsi Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: machine: x86_64 Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: clock source: unix Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: pcre jit disabled Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: detected number of CPU cores: 4 Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: current working directory: / Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: detected binary path: /usr/bin/uwsgi-core Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: your processes number limit is 47680 Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: your memory page size is 4096 bytes Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: detected max file descriptor number: 2048 Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: lock engine: pthread robust mutexes Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: thunder lock: enabled Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 11:28:45.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 11:28:45.064536 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: Python main interpreter initialized at 0x560bb99139c0 Aug 31 11:28:45.064536 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: python threads support enabled Aug 31 11:28:45.064536 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: your server socket listen backlog is limited to 100 connections Aug 31 11:28:45.064536 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: your mercy for graceful operations on workers is 90 seconds Aug 31 11:28:45.064933 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 11:28:45.065002 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: *** Operational MODE: preforking *** Aug 31 11:28:45.065253 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: *** uWSGI is running in multiple interpreter mode *** Aug 31 11:28:45.065253 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: spawned uWSGI master process (pid: 81603) Aug 31 11:28:45.065469 dv-cinder-839768-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 11:28:45.066311 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: spawned uWSGI worker 1 (pid: 81620, cores: 1) Aug 31 11:28:45.066639 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: spawned uWSGI worker 2 (pid: 81621, cores: 1) Aug 31 11:28:45.066788 dv-cinder-839768-3-iscsi devstack@keystone.service[81603]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 11:28:49.449976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 11:28:49.449976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: from cryptography.utils import int_from_bytes Aug 31 11:28:49.451263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 11:28:49.451263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: from cryptography.utils import int_from_bytes Aug 31 11:28:49.451750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 11:28:49.451750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: from cryptography.utils import int_from_bytes Aug 31 11:28:49.452199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 11:28:49.452199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: from cryptography.utils import int_from_bytes Aug 31 11:28:49.502850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 11:28:49.503036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 11:28:49.503212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] command line args: [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 11:28:49.503481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 11:28:49.503643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 11:28:49.503723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ================================================================================ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 11:28:49.503842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 11:28:49.503926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] admin_token = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.504038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] command line args: [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 11:28:49.504116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_dir = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.504308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.504392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 11:28:49.504503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ================================================================================ {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 11:28:49.504590 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_source = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.504700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] debug = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.504781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] admin_token = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.504902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.505197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] config_dir = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.505197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.505384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.505484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] config_source = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.505584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.505685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] debug = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.505787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.505886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.506014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.506114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.506114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.506296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.506296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_config_append = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.506478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.506571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.506668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.506778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_dir = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.506778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.506950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.507042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_config_append = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.507139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.507241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.507363 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.507437 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_dir = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.507550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.507628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_file = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.507748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.507829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.507954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.508029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.508141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.508214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.508326 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.508410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.508519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.508604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.508741 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.508857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.509000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.509091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.509201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.509283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.509437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.509562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.509688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.509772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.509930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.510006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.510186 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.510307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.510423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.510660 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.510796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.510866 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] publish_errors = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.510925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.511081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.511316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.511413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.511496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.511599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] publish_errors = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.511733 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.511853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.511972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.512048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.512211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] standard_threads = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.512301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.513072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.513301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.513659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.513756 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.513756 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.513883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] standard_threads = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.513957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_journal = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_json = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_stderr = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_syslog = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] use_journal = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] use_json = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.514806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] use_stderr = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] use_syslog = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.514934 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.515001 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 11:28:49.515076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.515229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.515306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.515472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.515548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.515742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.515829 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.515984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.516060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.516224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.516298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.516466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.516569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.516740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.516821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.516979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.517053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.517217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.517299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.517464 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.517547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.517704 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.517785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.517940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.518062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.518191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.518274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.518429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.518511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.518679 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.518763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.518930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.519005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.519170 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.519251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.519394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.519470 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.519643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.519725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.519904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.519985 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.520140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.520216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.520395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.520469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.520652 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.520727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.520887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.520965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.521120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.521238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.521356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.521433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.521605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.521681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.521838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.521912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.522071 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.522151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.522302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.522378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.522548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.522624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.522782 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.522856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.523019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.523096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.523265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.523345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.523498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.523573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.523736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.523811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.523973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.524048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.524206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.524281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.524439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.524565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.524698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.524772 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.524931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.525005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.525154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.525232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.525395 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.525471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.525627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.525704 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.525868 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.525951 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.526102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.526177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.526341 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.526419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.526576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.526652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.526812 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.526919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.527046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.527146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.527284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.527385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.527520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.527628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.527759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.527844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.528016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.528097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.528265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.528377 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.528511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.528780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.528859 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.529033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.529119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.529281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.529358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.529521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.529608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.529778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.529866 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.530018 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.530136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.530253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.530332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.530489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.530569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.530723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.530803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.530984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.531063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.531222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.531302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.531464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.531539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.531706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.531791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.531943 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.532020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.532182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.532262 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.532420 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.532514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.532704 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.532783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.532945 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.533024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.533180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.533256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.533419 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.533502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.533661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.533763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.533905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.533988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.534141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.534218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.534381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.534458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.534620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.534696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.534859 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.534942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.535096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.535173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.535336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.535417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.535582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.535660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.535831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.535907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.536072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.536147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.536311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.536389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.536579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.536660 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.536822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.536900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.537062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.537141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.537300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.537376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.537533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.537617 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.537765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.537858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.537998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.538075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.538235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.538311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.538466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.538549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.538699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.538775 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.538933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.539008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.539169 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.539252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.539405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.539482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.539649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.539738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.539889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.539975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.540143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.540238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.540387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.540471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.external = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.540642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.540761 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.540925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.541017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.541162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.541246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.541396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.external = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.541472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.password = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.541625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.token = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.541709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.541867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.541946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.542106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.542183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.542355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.542441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.password = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.542590 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.token = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.542667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.542834 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.542915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.543075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.543165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.543307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.543384 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.543546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.543623 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.543868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.543957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.544126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.544299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.544434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.544579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.544736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.544826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.544994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.545079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.545250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.545328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.545505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.545591 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.545762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.545840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.546000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.546076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.546242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.546326 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.546495 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.546573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.546755 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.546833 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.546998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.547084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.547246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.547323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.547504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.547583 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.547743 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.547828 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.547981 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.548064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.548235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.548313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.548499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.548594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.548736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.548814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.548988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.549065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.549225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.549303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.549479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.549555 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.549715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.549791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.549963 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.550041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.550201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.550283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.550451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.550528 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.550690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.550768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.550941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.551018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.551182 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.551258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.551425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.551505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.551643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.551721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.551891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.551968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.552124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.552202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.552363 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.552441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.552629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.552765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.552886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.553021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.553122 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.553257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.553366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.553491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.553727 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.553972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.554222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.554472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.554576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.554730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.554839 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.554979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.555078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.555253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.555330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.555494 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.555582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.555734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.555812 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.555969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.556046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.556208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.556292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.556439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.556534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.556703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.556780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.556947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.557036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.557190 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.557268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.557439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.557523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.557672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.557760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.557922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.558002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.558172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.558255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.558410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.558494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.558653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.558730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.558900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.558976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.559130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.559209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.559920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.560003 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.560177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.560255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.560430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.560571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.560698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.560808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.560951 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.561054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.561204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.561284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.561440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.561519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.561683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.561760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.561920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.561999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.562163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.562241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.562407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.562485 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.562653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.562732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.562897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.562975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.563136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.563213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.563377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.563455 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.563615 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.563692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.563848 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.563926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.564081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.564159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.564335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.564413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.564589 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.564668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.564840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.564920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.565079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.565157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.565319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.565396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.565553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.565884 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.566041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.566119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.566279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.566356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.566515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.566594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.566750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.566828 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.566982 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.567059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.567230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.567308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.567468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.567544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.567702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.567779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.567941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.568020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.568180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.568296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.568409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.568483 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.568670 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.568749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.568908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.568984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.569149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.569226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.569389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.569467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.569628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.569705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.569874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.569950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.570109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.570188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.570347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.570423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.570587 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.570664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.570824 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.570902 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.571063 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.571139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.571300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.571382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.571538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.571617 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.571787 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.571877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.572027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.572105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.572286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.572365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.572540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.572622 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.572789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.572867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.573031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.573110 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.573269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.573348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.573510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.573587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.573746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.573866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.573999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.574118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.574255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.574358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.574495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.574601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.574744 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.574846 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.575593 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.575686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.575934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.576107 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.576180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.576250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.576407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.576581 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.576659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.576729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.576916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.576994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.577167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.577253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.577412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.577491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.577658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.577737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.577900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] role.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.577986 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.578147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.578225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] role.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.578380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.578456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.578635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.578720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.578867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.578944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.579118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.579195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.579368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.579457 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.579619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.579697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.579868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.579944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.580120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.580263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.580374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.580532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.580636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.580782 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.580888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.581036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.581140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.581289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.581393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.581544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.581646 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.581783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.582039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.582116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.582278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.582354 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.582523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.582600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.582758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.582836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.582992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.583069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.583226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.583308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.583479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.583556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.583734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.583811 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.583983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.584061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.584223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.584301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.584457 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.584547 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.584714 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.584792 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.584950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.585032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.585185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.585262 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.585434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.585513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.585673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.585750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.585916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.585993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.586157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.586236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.586395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.586471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.586632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.586709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.586875 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.586954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.587121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.587197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.587357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.587433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.587594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.587671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.587831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.587909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.588114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.588191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.588361 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.588439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.588632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.588712 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.588878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.588965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.589117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.589202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.589360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.589438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.589599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.589677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.589847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.589934 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.590127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.590205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.590363 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.590449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.590634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.590862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.590942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.591210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.591354 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.591485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.connection = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.591737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.591813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.591993 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.592213 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.592293 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.592469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.592661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.592824 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.592924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.593073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.593154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.593342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.593420 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.593647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.593776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.593928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.594163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.594240 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.594409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.594641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.594719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.594799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 11:28:49.595090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.595355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.595813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.596258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.596770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.597188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.597653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.598002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 11:28:49.598189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 11:28:49.624821 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.625053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.625280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.628326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.628597 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.628833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.633815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.637118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.663310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.663541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.663706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.663941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.717350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.718183 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.765723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.765953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.766673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.766913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.921573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.921843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.922016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.922242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.922466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.922685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.922971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.923207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.923434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.923655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.923874 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.924093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.946054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.946168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.946307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.946396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.946512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.946610 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.946731 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.946814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.946943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.947026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.947154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.947246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.947365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.947448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.958775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.959242 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.971690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.971874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.971958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.972102 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.972185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.972327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.972408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.972572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.972656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.972789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.972872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.973002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.973086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.973217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 11:28:49.990255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:49.990462 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 11:28:49.990694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:49.990809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:49.990892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 11:28:49.991008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 11:28:49.991236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:49.991326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:49.991439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 11:28:49.991521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:49.991720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:49.991801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:49.991986 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:49.992077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:49.992530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:49.992799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.001593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.001873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.002141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.002238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.002421 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.002505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.002763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.003050 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.009342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.010023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.012207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.013067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.013350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.013500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.013633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.013785 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.013914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.014070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.014160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.014353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.014435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.014640 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.014721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.014904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.014993 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.015175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.015256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.015451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.015532 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.015839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.016119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.016390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.037918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.038826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.039361 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.039641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.040287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.040601 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.044695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.044976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.045255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.045942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.046232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.046518 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.051676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.051964 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.052853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.053138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.055873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.056164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.056445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.056753 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.057028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.057214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.057506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.057791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.058074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.058356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.058632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.058904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.058984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.059175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.059316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.059636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.075743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.076017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.077017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.077302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.079872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.081180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.082963 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.083259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.084353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.084681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.087546 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.088874 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.090741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.092120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.093820 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.094123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.094410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.094696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.094984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.095356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.095439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.095709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.096120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.096586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.096886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.097171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.111710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.112003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.112270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.112555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.113348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.113641 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.113924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.114191 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.119238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.120886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.121056 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.122671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.122763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.122946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.124212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.124507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.125595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.125880 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.126171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.126454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.126738 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.127191 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.127486 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.127886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.128181 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.128472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.138069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.138356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.138641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.138920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.139201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.139477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.139699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.139791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.139988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.140369 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.140687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.140976 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.141258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.141550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.155718 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.157520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.158718 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.160397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.161589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.163008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.163275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.164459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.164745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.164843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.165059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.166347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.166636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.166930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.171692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.173199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.173457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.174941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.176039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.176337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.176648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.176927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.177945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.178240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.178538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.178832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.185366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.185652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.185943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.187232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.187526 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.187809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.193036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.193321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.193601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.193875 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.194147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.194426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.194695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.194930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 11:28:50.195012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.195217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.195308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.195487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.195582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.195757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.195884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.196015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.196097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.196275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.196356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.196576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.196661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.196869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.197150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.197434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.197706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.197967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.198232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.198503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 11:28:50.231075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560bb99139c0 pid: 81621 (default app) Aug 31 11:28:50.233116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560bb99139c0 pid: 81620 (default app) Aug 31 11:28:50.234489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc6baf-3e18-4945-98dd-77713c47e872 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:28:50.234776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc6baf-3e18-4945-98dd-77713c47e872 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:28:50.235048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc6baf-3e18-4945-98dd-77713c47e872 None None] PATH_INFO: `/v3` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:28:50.239201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 791 bytes} [Wed Aug 31 11:28:50 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:28:56.865332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10fbdf-2c3e-4aa6-9846-fe09649c129c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:28:56.865865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10fbdf-2c3e-4aa6-9846-fe09649c129c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:28:56.865865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10fbdf-2c3e-4aa6-9846-fe09649c129c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:28:56.870202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1/2] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:28:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:28:56.876070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a97943-1bfd-429e-8e80-966693492bd3 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:28:56.876610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a97943-1bfd-429e-8e80-966693492bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:28:56.876771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a97943-1bfd-429e-8e80-966693492bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:28:56.946414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: INFO dbcounter [None req-a1a97943-1bfd-429e-8e80-966693492bd3 None None] Registered counter for database keystone Aug 31 11:28:56.964911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG oslo_db.sqlalchemy.engines [None req-a1a97943-1bfd-429e-8e80-966693492bd3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81621) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 11:28:56.971594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writer thread running {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 11:28:57.149162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG passlib.handlers.bcrypt [None req-a1a97943-1bfd-429e-8e80-966693492bd3 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81621) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 11:28:57.149557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG passlib.handlers.bcrypt [None req-a1a97943-1bfd-429e-8e80-966693492bd3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81621) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 11:28:57.207851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a1a97943-1bfd-429e-8e80-966693492bd3 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:28:57.209411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a1a97943-1bfd-429e-8e80-966693492bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:28:57.335317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 2/3] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:28:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 31 11:28:57.838895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10fbdf-2c3e-4aa6-9846-fe09649c129c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:28:57.907482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: INFO dbcounter [None req-be10fbdf-2c3e-4aa6-9846-fe09649c129c None None] Registered counter for database keystone Aug 31 11:28:57.928351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG oslo_db.sqlalchemy.engines [None req-be10fbdf-2c3e-4aa6-9846-fe09649c129c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81620) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 11:28:57.935025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writer thread running {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 11:28:58.055539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10fbdf-2c3e-4aa6-9846-fe09649c129c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:28:58.059969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec619cfd-f6e1-47bf-a687-f214eda3defb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:28:58.060628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619cfd-f6e1-47bf-a687-f214eda3defb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:28:58.060912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619cfd-f6e1-47bf-a687-f214eda3defb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:28:58.061186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619cfd-f6e1-47bf-a687-f214eda3defb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:28:58.069482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec619cfd-f6e1-47bf-a687-f214eda3defb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:28:58.120411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ec619cfd-f6e1-47bf-a687-f214eda3defb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 11:28:58.120787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec619cfd-f6e1-47bf-a687-f214eda3defb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:28:58.130347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 2/4] 10.222.0.60 () {58 vars in 1324 bytes} [Wed Aug 31 11:28:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:28:58.136298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a97943-1bfd-429e-8e80-966693492bd3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:28:58.151017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a97943-1bfd-429e-8e80-966693492bd3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:28:58.155965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dd9fa-c85d-4318-b3f5-0eb9aff820ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:28:58.157094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dd9fa-c85d-4318-b3f5-0eb9aff820ca admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:28:58.157376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dd9fa-c85d-4318-b3f5-0eb9aff820ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:28:58.157652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dd9fa-c85d-4318-b3f5-0eb9aff820ca admin admin] PATH_INFO: `/v3/projects/927949f45e9a418783820746376979d1` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:28:58.159223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dd9fa-c85d-4318-b3f5-0eb9aff820ca admin admin] RBAC: Authorizing `identity:get_project(project_id=927949f45e9a418783820746376979d1)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:28:58.214096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: WARNING keystone.common.rbac_enforcer.enforcer [None req-189dd9fa-c85d-4318-b3f5-0eb9aff820ca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 11:28:58.214391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dd9fa-c85d-4318-b3f5-0eb9aff820ca admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:28:58.219768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 3/5] 10.222.0.60 () {56 vars in 1207 bytes} [Wed Aug 31 11:28:58 2022] GET /identity/v3/projects/927949f45e9a418783820746376979d1 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:00.148367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c236237-1676-4bce-a5aa-ec82f9b2c3a6 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:00.148960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c236237-1676-4bce-a5aa-ec82f9b2c3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:00.148960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c236237-1676-4bce-a5aa-ec82f9b2c3a6 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:00.153608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 3/6] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:00.159018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbd0cc-4a45-49e8-946d-dd0d5e93afef None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:00.159414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbd0cc-4a45-49e8-946d-dd0d5e93afef None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:00.159560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbd0cc-4a45-49e8-946d-dd0d5e93afef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:00.179733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-24cbd0cc-4a45-49e8-946d-dd0d5e93afef None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:00.181271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-24cbd0cc-4a45-49e8-946d-dd0d5e93afef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:00.188993 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 4/7] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 11:29:00.462279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c236237-1676-4bce-a5aa-ec82f9b2c3a6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:00.474799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c236237-1676-4bce-a5aa-ec82f9b2c3a6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:00.478973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32287ae-caad-4c5f-8890-50c28c006aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:00.479568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32287ae-caad-4c5f-8890-50c28c006aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:00.479848 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32287ae-caad-4c5f-8890-50c28c006aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:00.480121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32287ae-caad-4c5f-8890-50c28c006aa5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:00.487718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32287ae-caad-4c5f-8890-50c28c006aa5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:00.489532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32287ae-caad-4c5f-8890-50c28c006aa5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:00.498322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 4/8] 10.222.0.60 () {58 vars in 1324 bytes} [Wed Aug 31 11:29:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:29:00.503472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbd0cc-4a45-49e8-946d-dd0d5e93afef None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:00.512997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbd0cc-4a45-49e8-946d-dd0d5e93afef None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:00.517330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93287ad0-f553-4ffa-a18e-74d94799d732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:00.517951 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93287ad0-f553-4ffa-a18e-74d94799d732 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:00.518234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93287ad0-f553-4ffa-a18e-74d94799d732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:00.518507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93287ad0-f553-4ffa-a18e-74d94799d732 admin admin] PATH_INFO: `/v3/users/824ec611f32e40609c56e94f7edd552e` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:00.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93287ad0-f553-4ffa-a18e-74d94799d732 admin admin] RBAC: Authorizing `identity:get_user(user_id=824ec611f32e40609c56e94f7edd552e)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:00.521674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93287ad0-f553-4ffa-a18e-74d94799d732 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:00.523479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 5/9] 10.222.0.60 () {56 vars in 1198 bytes} [Wed Aug 31 11:29:00 2022] GET /identity/v3/users/824ec611f32e40609c56e94f7edd552e => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:02.479700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb25658-a893-482a-818f-61e4d768a38c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:02.480259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb25658-a893-482a-818f-61e4d768a38c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:02.480259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb25658-a893-482a-818f-61e4d768a38c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:02.482041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 5/10] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:02.487212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec370-e158-4d48-b735-6b0b5bbec2dd None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:02.487611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec370-e158-4d48-b735-6b0b5bbec2dd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:02.487776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec370-e158-4d48-b735-6b0b5bbec2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:02.506178 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cc3ec370-e158-4d48-b735-6b0b5bbec2dd None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:02.507363 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cc3ec370-e158-4d48-b735-6b0b5bbec2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:02.511815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca791e6-a55a-4d08-90be-82719a6d6bf4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:02.512757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca791e6-a55a-4d08-90be-82719a6d6bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:02.513141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca791e6-a55a-4d08-90be-82719a6d6bf4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:02.514581 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 6/11] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:29:02.519482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0ac84-c965-4144-ace3-b335148fadd3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:02.519767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0ac84-c965-4144-ace3-b335148fadd3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:02.520044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0ac84-c965-4144-ace3-b335148fadd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:02.561808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG passlib.handlers.bcrypt [None req-04b0ac84-c965-4144-ace3-b335148fadd3 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81620) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 11:29:02.562213 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG passlib.handlers.bcrypt [None req-04b0ac84-c965-4144-ace3-b335148fadd3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81620) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 11:29:02.588234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 6/12] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:02.612142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-04b0ac84-c965-4144-ace3-b335148fadd3 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:02.613787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-04b0ac84-c965-4144-ace3-b335148fadd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:02.714127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 7/13] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:02.866078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ec370-e158-4d48-b735-6b0b5bbec2dd None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:02.892071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:02.971630 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ec370-e158-4d48-b735-6b0b5bbec2dd None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:02.990052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0ac84-c965-4144-ace3-b335148fadd3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:03.009324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bafe2-6693-4e36-95e3-b02483f404aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:03.009955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bafe2-6693-4e36-95e3-b02483f404aa None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:03.010241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bafe2-6693-4e36-95e3-b02483f404aa None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:03.010518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bafe2-6693-4e36-95e3-b02483f404aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:03.022496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bafe2-6693-4e36-95e3-b02483f404aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-855bafe2-6693-4e36-95e3-b02483f404aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:03.031530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:03.032631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:03.033459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 7/14] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:29:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:03.038481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bafe2-6693-4e36-95e3-b02483f404aa None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:03.091119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0ac84-c965-4144-ace3-b335148fadd3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:03.093789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:03.122094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:03.131998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735371ba-af09-4713-92c3-dccd735b533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:03.132990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735371ba-af09-4713-92c3-dccd735b533e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:03.133273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735371ba-af09-4713-92c3-dccd735b533e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:03.133634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735371ba-af09-4713-92c3-dccd735b533e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:03.143570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bafe2-6693-4e36-95e3-b02483f404aa None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:03.176666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb3d09-d914-4046-bffb-dd0defdac98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:03.177306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb3d09-d914-4046-bffb-dd0defdac98e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:03.177592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb3d09-d914-4046-bffb-dd0defdac98e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:03.177869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb3d09-d914-4046-bffb-dd0defdac98e None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:03.178572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb3d09-d914-4046-bffb-dd0defdac98e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:03.178905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb3d09-d914-4046-bffb-dd0defdac98e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:03.180283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb3d09-d914-4046-bffb-dd0defdac98e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:03.192584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 8/15] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:03.198031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb3d09-d914-4046-bffb-dd0defdac98e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:03.205406 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735371ba-af09-4713-92c3-dccd735b533e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:03.206915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735371ba-af09-4713-92c3-dccd735b533e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:03.297264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb3d09-d914-4046-bffb-dd0defdac98e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:03.303145 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 8/16] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 11:29:03.308724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735371ba-af09-4713-92c3-dccd735b533e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:03.312054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:03.331324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e902a-9559-46b6-8322-9c1f5c30bc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:03.331956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e902a-9559-46b6-8322-9c1f5c30bc29 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:03.332240 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e902a-9559-46b6-8322-9c1f5c30bc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:03.332545 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e902a-9559-46b6-8322-9c1f5c30bc29 None admin] PATH_INFO: `/v3/users/824ec611f32e40609c56e94f7edd552e` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:03.333691 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e902a-9559-46b6-8322-9c1f5c30bc29 None admin] RBAC: Authorizing `identity:get_user(user_id=824ec611f32e40609c56e94f7edd552e)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:03.335157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e902a-9559-46b6-8322-9c1f5c30bc29 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:03.336751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 9/17] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:03 2022] GET /identity/v3/users/824ec611f32e40609c56e94f7edd552e => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:03.342459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e902a-9559-46b6-8322-9c1f5c30bc29 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:03.358321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:03.407369 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735371ba-af09-4713-92c3-dccd735b533e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:03.441707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b247f-3c42-493f-b44d-df781a9efe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:03.442554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b247f-3c42-493f-b44d-df781a9efe52 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:03.442836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b247f-3c42-493f-b44d-df781a9efe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:03.443111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b247f-3c42-493f-b44d-df781a9efe52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:03.443592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e902a-9559-46b6-8322-9c1f5c30bc29 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:03.462522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b247f-3c42-493f-b44d-df781a9efe52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:03.463989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b247f-3c42-493f-b44d-df781a9efe52 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:03.478651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd8c35-8afd-4b24-a3ad-366a51f76fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:03.483998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8c35-8afd-4b24-a3ad-366a51f76fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:03.484301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8c35-8afd-4b24-a3ad-366a51f76fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:03.484616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8c35-8afd-4b24-a3ad-366a51f76fb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:03.485821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd8c35-8afd-4b24-a3ad-366a51f76fb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-fa1b247f-3c42-493f-b44d-df781a9efe52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:29:03.486841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:03.488058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:29:03.488925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:29:03.488925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:29:03.488925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:03.489077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd8c35-8afd-4b24-a3ad-366a51f76fb8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:03.489413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 9/18] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:29:03.489487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 10/19] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:03.493898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b247f-3c42-493f-b44d-df781a9efe52 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:03.494422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd8c35-8afd-4b24-a3ad-366a51f76fb8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:03.556193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:03.579487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:03.599851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd8c35-8afd-4b24-a3ad-366a51f76fb8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:03.600901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b247f-3c42-493f-b44d-df781a9efe52 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:03.633665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4e787-d58d-4579-a6c3-191d61bb7ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:03.634211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de5225-be64-42a4-af8c-de59927975c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:03.634344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4e787-d58d-4579-a6c3-191d61bb7ffd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:03.634544 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4e787-d58d-4579-a6c3-191d61bb7ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:03.634804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de5225-be64-42a4-af8c-de59927975c4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:03.634900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4e787-d58d-4579-a6c3-191d61bb7ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:03.635082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de5225-be64-42a4-af8c-de59927975c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:03.635351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de5225-be64-42a4-af8c-de59927975c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:03.635545 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4e787-d58d-4579-a6c3-191d61bb7ffd None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.domain.id=default, user.id=824ec611f32e40609c56e94f7edd552e) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:03.635858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4e787-d58d-4579-a6c3-191d61bb7ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:03.635983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de5225-be64-42a4-af8c-de59927975c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:03.636281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de5225-be64-42a4-af8c-de59927975c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:03.638169 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4e787-d58d-4579-a6c3-191d61bb7ffd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:03.638299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de5225-be64-42a4-af8c-de59927975c4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:03.645025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 11/20] 10.222.0.60 () {56 vars in 1322 bytes} [Wed Aug 31 11:29:03 2022] GET /identity/v3/role_assignments?user.id=824ec611f32e40609c56e94f7edd552e&scope.domain.id=default&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:03.657505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 10/21] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:05.823078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29e2ab-3519-4a9a-86bf-f6f48ec17d0f None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:05.823601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29e2ab-3519-4a9a-86bf-f6f48ec17d0f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:05.823688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29e2ab-3519-4a9a-86bf-f6f48ec17d0f None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:05.825851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 12/22] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:05.831418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b2602-0ca5-4889-804b-280af6954760 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:05.832249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b2602-0ca5-4889-804b-280af6954760 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:05.832759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b2602-0ca5-4889-804b-280af6954760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:05.858614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-660b2602-0ca5-4889-804b-280af6954760 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:05.860136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-660b2602-0ca5-4889-804b-280af6954760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:05.898215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdef9fd-7315-4797-823a-5d56f6ca160c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:05.898866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdef9fd-7315-4797-823a-5d56f6ca160c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:05.899352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdef9fd-7315-4797-823a-5d56f6ca160c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:05.902236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 13/23] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:05.907291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18378d5d-81a9-4642-b6f2-0c708a5f8c5e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:05.907834 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18378d5d-81a9-4642-b6f2-0c708a5f8c5e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:05.908333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18378d5d-81a9-4642-b6f2-0c708a5f8c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:05.945716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-18378d5d-81a9-4642-b6f2-0c708a5f8c5e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:05.947257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-18378d5d-81a9-4642-b6f2-0c708a5f8c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:05.979199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 11/24] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:06.065301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 14/25] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:06.235018 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fd659-75c6-424a-a3f0-466dfa8ab68f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:06.235662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fd659-75c6-424a-a3f0-466dfa8ab68f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:06.236144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fd659-75c6-424a-a3f0-466dfa8ab68f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:06.237978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 12/26] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:06.243021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6e845-d9a8-49a5-a946-f095a2764cb9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:06.243315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6e845-d9a8-49a5-a946-f095a2764cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:06.243596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6e845-d9a8-49a5-a946-f095a2764cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:06.268741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-80a6e845-d9a8-49a5-a946-f095a2764cb9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:06.269947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-80a6e845-d9a8-49a5-a946-f095a2764cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:06.359826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 15/27] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:06.507105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e150a3-f1b8-436d-8d70-0d024b378166 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:06.507721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e150a3-f1b8-436d-8d70-0d024b378166 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:06.508202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e150a3-f1b8-436d-8d70-0d024b378166 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:06.509758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 13/28] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:06.515460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fb33d-4ea9-414f-bf74-a1b5c4020ee5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:06.515752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fb33d-4ea9-414f-bf74-a1b5c4020ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:06.516034 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fb33d-4ea9-414f-bf74-a1b5c4020ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:06.527201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:06.540820 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-fc9fb33d-4ea9-414f-bf74-a1b5c4020ee5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:06.542084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fc9fb33d-4ea9-414f-bf74-a1b5c4020ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:06.561254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e150a3-f1b8-436d-8d70-0d024b378166 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:06.618989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 16/29] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:06.620356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fb33d-4ea9-414f-bf74-a1b5c4020ee5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:06.645604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:06.762345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e150a3-f1b8-436d-8d70-0d024b378166 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:06.779353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fb33d-4ea9-414f-bf74-a1b5c4020ee5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:06.829920 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc28b4-a80d-4565-bdee-5e6497c5b8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:06.830552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc28b4-a80d-4565-bdee-5e6497c5b8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:06.830831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc28b4-a80d-4565-bdee-5e6497c5b8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:06.831104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc28b4-a80d-4565-bdee-5e6497c5b8b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:06.845771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df5d64-8b80-4e5e-be2d-497c3481d4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:06.846422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df5d64-8b80-4e5e-be2d-497c3481d4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:06.846710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df5d64-8b80-4e5e-be2d-497c3481d4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:06.846989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df5d64-8b80-4e5e-be2d-497c3481d4c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:06.850676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:06.857100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc28b4-a80d-4565-bdee-5e6497c5b8b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:06.858590 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc28b4-a80d-4565-bdee-5e6497c5b8b4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:06.860784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df5d64-8b80-4e5e-be2d-497c3481d4c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-07df5d64-8b80-4e5e-be2d-497c3481d4c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:06.865778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:06.866628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:06.867972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 17/30] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:29:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:06.869350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df5d64-8b80-4e5e-be2d-497c3481d4c6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-40bc28b4-a80d-4565-bdee-5e6497c5b8b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:29:06.894807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:06.895812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:29:06.896634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:29:06.896634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:29:06.896634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:06.900864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 14/31] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:29:06.901697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc28b4-a80d-4565-bdee-5e6497c5b8b4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:07.043801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df5d64-8b80-4e5e-be2d-497c3481d4c6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:07.077749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:07.082181 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc28b4-a80d-4565-bdee-5e6497c5b8b4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:07.084246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bd0fc-4341-421d-ac34-5ea1d1cc6a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:07.085126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bd0fc-4341-421d-ac34-5ea1d1cc6a24 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:07.086285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bd0fc-4341-421d-ac34-5ea1d1cc6a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:07.086567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bd0fc-4341-421d-ac34-5ea1d1cc6a24 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:07.087255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bd0fc-4341-421d-ac34-5ea1d1cc6a24 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:07.088697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bd0fc-4341-421d-ac34-5ea1d1cc6a24 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:07.126156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5ea12-0be7-4dc7-853c-9b6b0348095a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:07.126930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5ea12-0be7-4dc7-853c-9b6b0348095a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:07.127298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5ea12-0be7-4dc7-853c-9b6b0348095a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:07.127575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5ea12-0be7-4dc7-853c-9b6b0348095a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:07.128431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5ea12-0be7-4dc7-853c-9b6b0348095a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:07.128764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5ea12-0be7-4dc7-853c-9b6b0348095a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:07.130217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5ea12-0be7-4dc7-853c-9b6b0348095a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:07.145456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 15/32] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:07.146393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5ea12-0be7-4dc7-853c-9b6b0348095a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:07.224843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 18/33] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Aug 31 11:29:06 2022] POST /identity/v3/roles => generated 225 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:07.225810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bd0fc-4341-421d-ac34-5ea1d1cc6a24 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:07.263452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5ea12-0be7-4dc7-853c-9b6b0348095a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:07.306693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915db08-6f15-4e35-9d9b-e59ba98022a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:07.307365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915db08-6f15-4e35-9d9b-e59ba98022a1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:07.307762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915db08-6f15-4e35-9d9b-e59ba98022a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:07.308415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915db08-6f15-4e35-9d9b-e59ba98022a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:07.310315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915db08-6f15-4e35-9d9b-e59ba98022a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:07.310627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915db08-6f15-4e35-9d9b-e59ba98022a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:07.312616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915db08-6f15-4e35-9d9b-e59ba98022a1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:07.327960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 16/34] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:07.328990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915db08-6f15-4e35-9d9b-e59ba98022a1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:07.356158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:07.383749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:07.455992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bd0fc-4341-421d-ac34-5ea1d1cc6a24 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:07.466441 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915db08-6f15-4e35-9d9b-e59ba98022a1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:07.493612 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600ed53-9713-4ba2-b459-a6a80f3f26a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:07.494497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600ed53-9713-4ba2-b459-a6a80f3f26a4 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:07.494605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600ed53-9713-4ba2-b459-a6a80f3f26a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:07.494841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600ed53-9713-4ba2-b459-a6a80f3f26a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:07.495572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600ed53-9713-4ba2-b459-a6a80f3f26a4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:07.497996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600ed53-9713-4ba2-b459-a6a80f3f26a4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:07.517558 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7be0e-0151-489a-bcbe-f4976b3288c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:07.517909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 19/35] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Aug 31 11:29:07 2022] POST /identity/v3/roles => generated 219 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:07.518293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7be0e-0151-489a-bcbe-f4976b3288c1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:07.518575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7be0e-0151-489a-bcbe-f4976b3288c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:07.518736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600ed53-9713-4ba2-b459-a6a80f3f26a4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:07.518832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7be0e-0151-489a-bcbe-f4976b3288c1 None admin] PATH_INFO: `/v3/users/824ec611f32e40609c56e94f7edd552e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:07.520555 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7be0e-0151-489a-bcbe-f4976b3288c1 None admin] RBAC: Authorizing `identity:get_user(user_id=824ec611f32e40609c56e94f7edd552e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:07.522283 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7be0e-0151-489a-bcbe-f4976b3288c1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:07.524327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 17/36] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:07 2022] GET /identity/v3/users/824ec611f32e40609c56e94f7edd552e => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:07.525659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50c9b9-407a-48c7-8978-56a9870f3a10 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:07.525908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50c9b9-407a-48c7-8978-56a9870f3a10 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:07.526180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50c9b9-407a-48c7-8978-56a9870f3a10 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:07.529660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 18/37] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:07.533674 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50c9b9-407a-48c7-8978-56a9870f3a10 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:07.662461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:07.666937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600ed53-9713-4ba2-b459-a6a80f3f26a4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:07.684834 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:07.686248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50c9b9-407a-48c7-8978-56a9870f3a10 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:07.714693 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bc89e-0d18-45be-b8e8-ec9f43941b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:07.715355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bc89e-0d18-45be-b8e8-ec9f43941b24 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:07.715651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bc89e-0d18-45be-b8e8-ec9f43941b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:07.716116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bc89e-0d18-45be-b8e8-ec9f43941b24 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:07.716864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bc89e-0d18-45be-b8e8-ec9f43941b24 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:07.718349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bc89e-0d18-45be-b8e8-ec9f43941b24 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:07.732838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff39a7f-99e4-4d78-b3e3-de12276235bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:07.733769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff39a7f-99e4-4d78-b3e3-de12276235bb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:07.733769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff39a7f-99e4-4d78-b3e3-de12276235bb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:07.733928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff39a7f-99e4-4d78-b3e3-de12276235bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:07.735511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff39a7f-99e4-4d78-b3e3-de12276235bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:07.737102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff39a7f-99e4-4d78-b3e3-de12276235bb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:07.743898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 19/38] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:07.743898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f02731-7f66-4a06-bd90-13f034279d5b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:07.743898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f02731-7f66-4a06-bd90-13f034279d5b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:07.743898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f02731-7f66-4a06-bd90-13f034279d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:07.754605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 20/39] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:29:07 2022] POST /identity/v3/projects => generated 301 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:07.755782 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b4058-19f3-41ca-b863-f98755046947 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:07.756071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b4058-19f3-41ca-b863-f98755046947 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:07.756367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b4058-19f3-41ca-b863-f98755046947 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:07.757760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 21/40] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:07.758416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b4058-19f3-41ca-b863-f98755046947 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:07.765411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-93f02731-7f66-4a06-bd90-13f034279d5b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:07.766989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-93f02731-7f66-4a06-bd90-13f034279d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:07.852543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 20/41] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:07.852543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a932-3924-4014-89c8-9e0fc761763a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:07.852543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a932-3924-4014-89c8-9e0fc761763a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:07.852543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a932-3924-4014-89c8-9e0fc761763a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:07.881517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5364a932-3924-4014-89c8-9e0fc761763a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:07.883947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5364a932-3924-4014-89c8-9e0fc761763a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:07.892618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b4058-19f3-41ca-b863-f98755046947 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:07.976299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 21/42] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:08.040341 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7596c42-2c23-4531-8924-e3f2c911fc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:08.041031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7596c42-2c23-4531-8924-e3f2c911fc98 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:08.041318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7596c42-2c23-4531-8924-e3f2c911fc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:08.041598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7596c42-2c23-4531-8924-e3f2c911fc98 None admin] PATH_INFO: `/v3/domains/default/users/824ec611f32e40609c56e94f7edd552e/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:08.044052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7596c42-2c23-4531-8924-e3f2c911fc98 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=824ec611f32e40609c56e94f7edd552e, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:08.045530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7596c42-2c23-4531-8924-e3f2c911fc98 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:08.060840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 22/43] 10.222.0.60 () {58 vars in 1361 bytes} [Wed Aug 31 11:29:07 2022] PUT /identity/v3/domains/default/users/824ec611f32e40609c56e94f7edd552e/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:08.132101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a932-3924-4014-89c8-9e0fc761763a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:08.154490 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:08.248304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a932-3924-4014-89c8-9e0fc761763a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:08.274202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7596c42-2c23-4531-8924-e3f2c911fc98 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:08.282766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda49a81-dc91-487a-ad8d-4112cde945a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:08.283403 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda49a81-dc91-487a-ad8d-4112cde945a9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:08.283685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda49a81-dc91-487a-ad8d-4112cde945a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:08.283957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda49a81-dc91-487a-ad8d-4112cde945a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:08.285091 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda49a81-dc91-487a-ad8d-4112cde945a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:08.286535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda49a81-dc91-487a-ad8d-4112cde945a9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:08.288290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 22/44] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:08.293474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda49a81-dc91-487a-ad8d-4112cde945a9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:08.303636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:08.384707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7596c42-2c23-4531-8924-e3f2c911fc98 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:08.392780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:08.411235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda49a81-dc91-487a-ad8d-4112cde945a9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:08.420649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed85b1-9e94-4041-a7e3-361f95a8ab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:08.421261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed85b1-9e94-4041-a7e3-361f95a8ab99 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:08.421545 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed85b1-9e94-4041-a7e3-361f95a8ab99 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:08.421818 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed85b1-9e94-4041-a7e3-361f95a8ab99 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:08.422504 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed85b1-9e94-4041-a7e3-361f95a8ab99 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:08.423897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed85b1-9e94-4041-a7e3-361f95a8ab99 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:08.438408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 23/45] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Aug 31 11:29:08 2022] POST /identity/v3/roles => generated 223 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:08.451509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0d02-c2e4-42ae-ac31-bde524186125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:08.452316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0d02-c2e4-42ae-ac31-bde524186125 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:08.452424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0d02-c2e4-42ae-ac31-bde524186125 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:08.452736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0d02-c2e4-42ae-ac31-bde524186125 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:08.453529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0d02-c2e4-42ae-ac31-bde524186125 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:08.455226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0d02-c2e4-42ae-ac31-bde524186125 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:08.594469 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 23/46] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:29:08 2022] POST /identity/v3/projects => generated 312 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:29:10.007760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539deb5-4a70-405c-8a69-d5ffe29ed086 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:10.008268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539deb5-4a70-405c-8a69-d5ffe29ed086 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:10.008268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539deb5-4a70-405c-8a69-d5ffe29ed086 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:10.009637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 24/47] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:10.016127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a18a6-b10b-4f94-8b64-57bce43749bf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:10.016572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a18a6-b10b-4f94-8b64-57bce43749bf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:10.016727 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a18a6-b10b-4f94-8b64-57bce43749bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:10.035920 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6a3a18a6-b10b-4f94-8b64-57bce43749bf None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:10.037452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a3a18a6-b10b-4f94-8b64-57bce43749bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:10.102815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 24/48] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:10.371651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539deb5-4a70-405c-8a69-d5ffe29ed086 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:10.456602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:10.473843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539deb5-4a70-405c-8a69-d5ffe29ed086 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:10.507244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759192f-1a5e-4327-97b3-a8fdd644dc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:10.507873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759192f-1a5e-4327-97b3-a8fdd644dc90 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:10.508159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759192f-1a5e-4327-97b3-a8fdd644dc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:10.508437 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759192f-1a5e-4327-97b3-a8fdd644dc90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:10.514479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12a93a-92c8-4236-86db-a2e917552b45 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:10.514768 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12a93a-92c8-4236-86db-a2e917552b45 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:10.515044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12a93a-92c8-4236-86db-a2e917552b45 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:10.516385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 25/49] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:29:10.518772 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759192f-1a5e-4327-97b3-a8fdd644dc90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:10.521226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acd02f-7b13-48ce-8d33-7e3bd6cca459 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:10.521577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acd02f-7b13-48ce-8d33-7e3bd6cca459 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:10.521921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acd02f-7b13-48ce-8d33-7e3bd6cca459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-3759192f-1a5e-4327-97b3-a8fdd644dc90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:10.523958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:10.524960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:10.525893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 25/50] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:29:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:10.533617 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759192f-1a5e-4327-97b3-a8fdd644dc90 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:10.543456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d8acd02f-7b13-48ce-8d33-7e3bd6cca459 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:10.544636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d8acd02f-7b13-48ce-8d33-7e3bd6cca459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:10.636116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759192f-1a5e-4327-97b3-a8fdd644dc90 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:10.672477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871df41-3d78-4849-89d2-cee679f46171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:10.673131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871df41-3d78-4849-89d2-cee679f46171 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:10.673416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871df41-3d78-4849-89d2-cee679f46171 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:10.673732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871df41-3d78-4849-89d2-cee679f46171 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:10.674427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871df41-3d78-4849-89d2-cee679f46171 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:10.674805 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871df41-3d78-4849-89d2-cee679f46171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:10.676261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871df41-3d78-4849-89d2-cee679f46171 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:10.687293 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 26/51] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:10.692930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871df41-3d78-4849-89d2-cee679f46171 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:10.704032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:10.711041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:10.720543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 26/52] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:10.791747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871df41-3d78-4849-89d2-cee679f46171 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:10.823892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c05439-04c1-402e-af3f-c876190f3fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:10.825131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c05439-04c1-402e-af3f-c876190f3fee None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:10.825578 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c05439-04c1-402e-af3f-c876190f3fee None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:10.825855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c05439-04c1-402e-af3f-c876190f3fee None admin] PATH_INFO: `/v3/users/824ec611f32e40609c56e94f7edd552e` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:10.826989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c05439-04c1-402e-af3f-c876190f3fee None admin] RBAC: Authorizing `identity:get_user(user_id=824ec611f32e40609c56e94f7edd552e)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:10.828430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c05439-04c1-402e-af3f-c876190f3fee None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:10.830019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 27/53] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:10 2022] GET /identity/v3/users/824ec611f32e40609c56e94f7edd552e => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:10.835580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acd02f-7b13-48ce-8d33-7e3bd6cca459 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:10.933617 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acd02f-7b13-48ce-8d33-7e3bd6cca459 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:10.967979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493fcdc-4f60-45d9-81e4-074e8bbc5b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:10.968645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493fcdc-4f60-45d9-81e4-074e8bbc5b43 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:10.968933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493fcdc-4f60-45d9-81e4-074e8bbc5b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:10.969207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493fcdc-4f60-45d9-81e4-074e8bbc5b43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:10.970304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493fcdc-4f60-45d9-81e4-074e8bbc5b43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:10.971870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493fcdc-4f60-45d9-81e4-074e8bbc5b43 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:10.973457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 27/54] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:10.979268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c05439-04c1-402e-af3f-c876190f3fee None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:11.000719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493fcdc-4f60-45d9-81e4-074e8bbc5b43 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:11.066886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:11.071585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:11.093550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c05439-04c1-402e-af3f-c876190f3fee None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:11.119230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493fcdc-4f60-45d9-81e4-074e8bbc5b43 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:11.128119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb9a6b-14db-4a87-9b6e-f4e78e1decc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:11.128780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb9a6b-14db-4a87-9b6e-f4e78e1decc2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:11.129128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb9a6b-14db-4a87-9b6e-f4e78e1decc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:11.129430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb9a6b-14db-4a87-9b6e-f4e78e1decc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:11.130524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb9a6b-14db-4a87-9b6e-f4e78e1decc2 None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.domain.id=default, user.id=824ec611f32e40609c56e94f7edd552e) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:11.131040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb9a6b-14db-4a87-9b6e-f4e78e1decc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:11.132821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb9a6b-14db-4a87-9b6e-f4e78e1decc2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:11.139437 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 28/55] 10.222.0.60 () {56 vars in 1322 bytes} [Wed Aug 31 11:29:10 2022] GET /identity/v3/role_assignments?user.id=824ec611f32e40609c56e94f7edd552e&scope.domain.id=default&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 530 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:11.165918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00605d47-67b3-4c71-a4c8-c1741e42c199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:11.166786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00605d47-67b3-4c71-a4c8-c1741e42c199 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:11.167074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00605d47-67b3-4c71-a4c8-c1741e42c199 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:11.167353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00605d47-67b3-4c71-a4c8-c1741e42c199 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:11.168652 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00605d47-67b3-4c71-a4c8-c1741e42c199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:11.170205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00605d47-67b3-4c71-a4c8-c1741e42c199 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:11.173057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 28/56] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:11.178802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb9a6b-14db-4a87-9b6e-f4e78e1decc2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:11.294319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb9a6b-14db-4a87-9b6e-f4e78e1decc2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:11.333860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f710f-b7be-48e0-a3e8-4928b4fd5fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:11.334781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f710f-b7be-48e0-a3e8-4928b4fd5fed None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:11.335071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f710f-b7be-48e0-a3e8-4928b4fd5fed None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:11.335353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f710f-b7be-48e0-a3e8-4928b4fd5fed None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:11.336067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f710f-b7be-48e0-a3e8-4928b4fd5fed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:11.338177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f710f-b7be-48e0-a3e8-4928b4fd5fed None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:11.352143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:11.368741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 29/57] 10.222.0.60 () {60 vars in 1140 bytes} [Wed Aug 31 11:29:11 2022] POST /identity/v3/projects => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:13.348407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11349866-bd8f-48aa-b4d2-d89761caf0c4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:13.348954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11349866-bd8f-48aa-b4d2-d89761caf0c4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:13.349039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11349866-bd8f-48aa-b4d2-d89761caf0c4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:13.350383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 29/58] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:13.356196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc48c83-45d8-4b5f-bf37-c33b171c5b26 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:13.356939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc48c83-45d8-4b5f-bf37-c33b171c5b26 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:13.356939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc48c83-45d8-4b5f-bf37-c33b171c5b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:13.375190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-dcc48c83-45d8-4b5f-bf37-c33b171c5b26 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:13.376746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dcc48c83-45d8-4b5f-bf37-c33b171c5b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:13.441565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 30/59] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:13.710932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11349866-bd8f-48aa-b4d2-d89761caf0c4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:13.813861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11349866-bd8f-48aa-b4d2-d89761caf0c4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:13.844382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:13.849539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0386d01-4785-4d88-98f9-8cbb91706609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:13.850160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0386d01-4785-4d88-98f9-8cbb91706609 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:13.850442 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0386d01-4785-4d88-98f9-8cbb91706609 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:13.850718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0386d01-4785-4d88-98f9-8cbb91706609 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:13.851818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0386d01-4785-4d88-98f9-8cbb91706609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:13.853296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0386d01-4785-4d88-98f9-8cbb91706609 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:13.854844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 30/60] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:13.860376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc48c83-45d8-4b5f-bf37-c33b171c5b26 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:13.955380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc48c83-45d8-4b5f-bf37-c33b171c5b26 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:13.977072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:13.988930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e6f48-1cb8-4fe6-8105-4cd4150a5124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:13.989532 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e6f48-1cb8-4fe6-8105-4cd4150a5124 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:13.989814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e6f48-1cb8-4fe6-8105-4cd4150a5124 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:13.990084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e6f48-1cb8-4fe6-8105-4cd4150a5124 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:13.990722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e6f48-1cb8-4fe6-8105-4cd4150a5124 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:13.992164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e6f48-1cb8-4fe6-8105-4cd4150a5124 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:14.018109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 31/61] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:29:13 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:17.294685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85de233-ba5d-4e47-98fa-8d9b4858333d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.295201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85de233-ba5d-4e47-98fa-8d9b4858333d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.295305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85de233-ba5d-4e47-98fa-8d9b4858333d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.299048 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 31/62] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:17.299161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9a7a-546b-4603-98fd-19042175b159 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.299161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9a7a-546b-4603-98fd-19042175b159 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.299161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9a7a-546b-4603-98fd-19042175b159 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.299161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 32/63] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:17.301655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41273e5f-c20f-4241-bb8b-96d71abec723 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.301951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41273e5f-c20f-4241-bb8b-96d71abec723 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.302232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41273e5f-c20f-4241-bb8b-96d71abec723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.305580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ed523-b299-4c11-b41f-42dedac63625 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.306083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ed523-b299-4c11-b41f-42dedac63625 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.306589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ed523-b299-4c11-b41f-42dedac63625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.335970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-41273e5f-c20f-4241-bb8b-96d71abec723 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:17.337208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-41273e5f-c20f-4241-bb8b-96d71abec723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:17.346071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5b6ed523-b299-4c11-b41f-42dedac63625 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:17.347270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5b6ed523-b299-4c11-b41f-42dedac63625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:17.426130 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 33/64] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:17.427803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25ccff-9fc4-4eab-8603-33e438ec1728 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.428310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25ccff-9fc4-4eab-8603-33e438ec1728 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.428801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25ccff-9fc4-4eab-8603-33e438ec1728 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.430474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 34/65] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:17.431855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1e8a8-925d-4235-bc71-8debb7c27109 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.432328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1e8a8-925d-4235-bc71-8debb7c27109 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.432809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1e8a8-925d-4235-bc71-8debb7c27109 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.434707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 35/66] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:17.437437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 32/67] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:17.440626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e02ae-f1dc-4fdb-bd09-0edcb1a2e326 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.441170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473128cc-dded-4b43-8718-9d15ea952911 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.441518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e02ae-f1dc-4fdb-bd09-0edcb1a2e326 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.441625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473128cc-dded-4b43-8718-9d15ea952911 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.441998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473128cc-dded-4b43-8718-9d15ea952911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.442149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e02ae-f1dc-4fdb-bd09-0edcb1a2e326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.461395 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a69e02ae-f1dc-4fdb-bd09-0edcb1a2e326 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:17.463083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a69e02ae-f1dc-4fdb-bd09-0edcb1a2e326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:17.478206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-473128cc-dded-4b43-8718-9d15ea952911 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:17.479708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-473128cc-dded-4b43-8718-9d15ea952911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:17.578335 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 33/68] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:17.581098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5b20e-219a-40aa-859b-471b3d88d890 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.581878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5b20e-219a-40aa-859b-471b3d88d890 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.582621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5b20e-219a-40aa-859b-471b3d88d890 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.585679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 34/69] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:17.595166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605c21f-7ad2-4b37-997c-d0ff88a58802 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.595744 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605c21f-7ad2-4b37-997c-d0ff88a58802 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.596310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605c21f-7ad2-4b37-997c-d0ff88a58802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.606079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 36/70] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:17.615541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8605c21f-7ad2-4b37-997c-d0ff88a58802 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:17.616996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8605c21f-7ad2-4b37-997c-d0ff88a58802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:17.702002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:17.713899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 35/71] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:17.757549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e02ae-f1dc-4fdb-bd09-0edcb1a2e326 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:17.803495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605c21f-7ad2-4b37-997c-d0ff88a58802 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:17.811773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:17.892203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e02ae-f1dc-4fdb-bd09-0edcb1a2e326 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:17.933395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605c21f-7ad2-4b37-997c-d0ff88a58802 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:17.937102 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a37876-424d-4c8b-b82d-5eb843884d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:17.937737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a37876-424d-4c8b-b82d-5eb843884d22 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.938018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a37876-424d-4c8b-b82d-5eb843884d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.938291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a37876-424d-4c8b-b82d-5eb843884d22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.953628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a37876-424d-4c8b-b82d-5eb843884d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-f4a37876-424d-4c8b-b82d-5eb843884d22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:17.960315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:17.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:17.962199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:17.964512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 37/72] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:17.966118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a37876-424d-4c8b-b82d-5eb843884d22 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:17.973124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df378b0-c6f9-488b-bb8a-3c34c666ac43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:17.973755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df378b0-c6f9-488b-bb8a-3c34c666ac43 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:17.974035 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df378b0-c6f9-488b-bb8a-3c34c666ac43 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:17.974303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df378b0-c6f9-488b-bb8a-3c34c666ac43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:17.987856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df378b0-c6f9-488b-bb8a-3c34c666ac43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2df378b0-c6f9-488b-bb8a-3c34c666ac43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:17.994144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:17.994994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:17.995745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 36/73] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:29:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:29:17.997186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df378b0-c6f9-488b-bb8a-3c34c666ac43 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.053549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:18.080720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:18.085145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a37876-424d-4c8b-b82d-5eb843884d22 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.107270 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df378b0-c6f9-488b-bb8a-3c34c666ac43 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.118867 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009afb02-e9b2-4d6f-a38a-01408c1f8e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.119493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009afb02-e9b2-4d6f-a38a-01408c1f8e05 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.119774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009afb02-e9b2-4d6f-a38a-01408c1f8e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.120048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009afb02-e9b2-4d6f-a38a-01408c1f8e05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.130995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009afb02-e9b2-4d6f-a38a-01408c1f8e05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-009afb02-e9b2-4d6f-a38a-01408c1f8e05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:18.136048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:18.137661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:18.138634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 38/74] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:18.140448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009afb02-e9b2-4d6f-a38a-01408c1f8e05 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.149917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08333b1-4b66-42e8-b7cb-44e5a2f52c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.150466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08333b1-4b66-42e8-b7cb-44e5a2f52c77 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.150749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08333b1-4b66-42e8-b7cb-44e5a2f52c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.151028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08333b1-4b66-42e8-b7cb-44e5a2f52c77 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.151703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08333b1-4b66-42e8-b7cb-44e5a2f52c77 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:18.152007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08333b1-4b66-42e8-b7cb-44e5a2f52c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:18.153594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08333b1-4b66-42e8-b7cb-44e5a2f52c77 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:18.161779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 37/75] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:18.162561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08333b1-4b66-42e8-b7cb-44e5a2f52c77 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.240671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009afb02-e9b2-4d6f-a38a-01408c1f8e05 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.263634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08333b1-4b66-42e8-b7cb-44e5a2f52c77 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.275834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:18.276157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa4090-7753-4961-90f5-ced6efcda0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.276894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa4090-7753-4961-90f5-ced6efcda0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.277226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa4090-7753-4961-90f5-ced6efcda0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.277592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa4090-7753-4961-90f5-ced6efcda0e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.287523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa4090-7753-4961-90f5-ced6efcda0e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-dbfa4090-7753-4961-90f5-ced6efcda0e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:18.292375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:18.293929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 39/76] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:29:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:18.294779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa4090-7753-4961-90f5-ced6efcda0e6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.302096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171914b6-2eed-416a-8aa6-ca528c949683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.302706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171914b6-2eed-416a-8aa6-ca528c949683 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.302990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171914b6-2eed-416a-8aa6-ca528c949683 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.303266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171914b6-2eed-416a-8aa6-ca528c949683 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.303941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171914b6-2eed-416a-8aa6-ca528c949683 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:18.304248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171914b6-2eed-416a-8aa6-ca528c949683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:18.305645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171914b6-2eed-416a-8aa6-ca528c949683 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:18.313967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 38/77] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:18.314487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:18.314998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171914b6-2eed-416a-8aa6-ca528c949683 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.399761 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa4090-7753-4961-90f5-ced6efcda0e6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.416245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171914b6-2eed-416a-8aa6-ca528c949683 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.440180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bb976-b2ff-413b-971a-3fdd96a4d263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.441124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bb976-b2ff-413b-971a-3fdd96a4d263 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.441418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bb976-b2ff-413b-971a-3fdd96a4d263 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.441698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bb976-b2ff-413b-971a-3fdd96a4d263 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.442847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bb976-b2ff-413b-971a-3fdd96a4d263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:18.444333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bb976-b2ff-413b-971a-3fdd96a4d263 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:18.446138 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 40/78] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:18.446798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bb976-b2ff-413b-971a-3fdd96a4d263 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.454185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d72310-61c8-46e9-8856-204f80c58d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.454788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d72310-61c8-46e9-8856-204f80c58d69 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.455067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d72310-61c8-46e9-8856-204f80c58d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.455340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d72310-61c8-46e9-8856-204f80c58d69 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.455996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d72310-61c8-46e9-8856-204f80c58d69 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:18.456297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d72310-61c8-46e9-8856-204f80c58d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:18.457680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d72310-61c8-46e9-8856-204f80c58d69 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:18.465704 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 39/79] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:18.469371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d72310-61c8-46e9-8856-204f80c58d69 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.505049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:18.535064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:18.550222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bb976-b2ff-413b-971a-3fdd96a4d263 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.570661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d72310-61c8-46e9-8856-204f80c58d69 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.584765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa6b2a-e262-45c1-80f3-f7148f31225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.585398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa6b2a-e262-45c1-80f3-f7148f31225f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.585686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa6b2a-e262-45c1-80f3-f7148f31225f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.585969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa6b2a-e262-45c1-80f3-f7148f31225f None admin] PATH_INFO: `/v3/users/8e67b785954f4fd4b095552017407c73` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.600098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa6b2a-e262-45c1-80f3-f7148f31225f None admin] RBAC: Authorizing `identity:get_user(user_id=8e67b785954f4fd4b095552017407c73)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:18.601618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa6b2a-e262-45c1-80f3-f7148f31225f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:18.603237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 41/80] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:18 2022] GET /identity/v3/users/8e67b785954f4fd4b095552017407c73 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:18.604010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa6b2a-e262-45c1-80f3-f7148f31225f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.610017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d5672-cb98-4842-ba40-6f4822925485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.610631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d5672-cb98-4842-ba40-6f4822925485 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.610916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d5672-cb98-4842-ba40-6f4822925485 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.611196 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d5672-cb98-4842-ba40-6f4822925485 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.611871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d5672-cb98-4842-ba40-6f4822925485 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:18.612179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d5672-cb98-4842-ba40-6f4822925485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:18.613596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d5672-cb98-4842-ba40-6f4822925485 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:18.622912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 40/81] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:29:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:18.623702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d5672-cb98-4842-ba40-6f4822925485 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.710133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa6b2a-e262-45c1-80f3-f7148f31225f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.721447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:18.725548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d5672-cb98-4842-ba40-6f4822925485 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.747343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c84ed-248d-43a1-bcdb-7a8d5d1463b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.747980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c84ed-248d-43a1-bcdb-7a8d5d1463b8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.748272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c84ed-248d-43a1-bcdb-7a8d5d1463b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.748588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c84ed-248d-43a1-bcdb-7a8d5d1463b8 None admin] PATH_INFO: `/v3/users/824ec611f32e40609c56e94f7edd552e` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.749726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c84ed-248d-43a1-bcdb-7a8d5d1463b8 None admin] RBAC: Authorizing `identity:get_user(user_id=824ec611f32e40609c56e94f7edd552e)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:18.751206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c84ed-248d-43a1-bcdb-7a8d5d1463b8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:18.752993 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 42/82] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:18 2022] GET /identity/v3/users/824ec611f32e40609c56e94f7edd552e => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:18.753600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c84ed-248d-43a1-bcdb-7a8d5d1463b8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.764299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0224bd5-daf3-42f2-b9f5-f1e01d9c4a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.764928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0224bd5-daf3-42f2-b9f5-f1e01d9c4a50 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.765217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0224bd5-daf3-42f2-b9f5-f1e01d9c4a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.765497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0224bd5-daf3-42f2-b9f5-f1e01d9c4a50 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.766150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0224bd5-daf3-42f2-b9f5-f1e01d9c4a50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:18.767588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0224bd5-daf3-42f2-b9f5-f1e01d9c4a50 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:18.771456 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:18.802458 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 41/83] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:29:18 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:18.803298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0224bd5-daf3-42f2-b9f5-f1e01d9c4a50 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.872181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c84ed-248d-43a1-bcdb-7a8d5d1463b8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.911656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb22a5-f75b-442b-a075-cab0e5164b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.912311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb22a5-f75b-442b-a075-cab0e5164b1d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.912634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb22a5-f75b-442b-a075-cab0e5164b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.912917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb22a5-f75b-442b-a075-cab0e5164b1d None admin] PATH_INFO: `/v3/users/8e67b785954f4fd4b095552017407c73` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.914155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb22a5-f75b-442b-a075-cab0e5164b1d None admin] RBAC: Authorizing `identity:get_user(user_id=8e67b785954f4fd4b095552017407c73)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:18.915652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb22a5-f75b-442b-a075-cab0e5164b1d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:18.916618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0224bd5-daf3-42f2-b9f5-f1e01d9c4a50 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:18.917505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 43/84] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:18 2022] GET /identity/v3/users/8e67b785954f4fd4b095552017407c73 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:18.918828 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb22a5-f75b-442b-a075-cab0e5164b1d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:18.959923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b5596-bf5e-40cf-8642-e1b6878b594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:18.960399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b5596-bf5e-40cf-8642-e1b6878b594c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:18.961324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b5596-bf5e-40cf-8642-e1b6878b594c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:18.961324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b5596-bf5e-40cf-8642-e1b6878b594c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:18.993687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:19.015916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:19.038643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb22a5-f75b-442b-a075-cab0e5164b1d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:19.059523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b5596-bf5e-40cf-8642-e1b6878b594c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:19.062130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b5596-bf5e-40cf-8642-e1b6878b594c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:19.078238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba54cea-0223-42d4-b757-faef3ea2a6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:19.078979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba54cea-0223-42d4-b757-faef3ea2a6ea None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:19.079271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba54cea-0223-42d4-b757-faef3ea2a6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:19.079548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba54cea-0223-42d4-b757-faef3ea2a6ea None admin] PATH_INFO: `/v3/users/8e67b785954f4fd4b095552017407c73` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:19.080729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba54cea-0223-42d4-b757-faef3ea2a6ea None admin] RBAC: Authorizing `identity:get_user(user_id=8e67b785954f4fd4b095552017407c73)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:19.082197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba54cea-0223-42d4-b757-faef3ea2a6ea None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:19.083807 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 44/85] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:18 2022] GET /identity/v3/users/8e67b785954f4fd4b095552017407c73 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:19.084613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba54cea-0223-42d4-b757-faef3ea2a6ea None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:19.194788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 42/86] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:19.195742 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b5596-bf5e-40cf-8642-e1b6878b594c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:19.226775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba54cea-0223-42d4-b757-faef3ea2a6ea None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:19.263441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009898a9-41ae-4eac-8309-fa8c4acd1154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:19.264063 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009898a9-41ae-4eac-8309-fa8c4acd1154 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:19.264344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009898a9-41ae-4eac-8309-fa8c4acd1154 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:19.264749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:19.264978 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009898a9-41ae-4eac-8309-fa8c4acd1154 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:19.286264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:19.323885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b5596-bf5e-40cf-8642-e1b6878b594c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:19.341212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009898a9-41ae-4eac-8309-fa8c4acd1154 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:19.342708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009898a9-41ae-4eac-8309-fa8c4acd1154 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:19.369537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc32858-ffe0-49ed-bd1f-9cbdbf9e8a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:19.370223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc32858-ffe0-49ed-bd1f-9cbdbf9e8a5e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:19.370759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc32858-ffe0-49ed-bd1f-9cbdbf9e8a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:19.371104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc32858-ffe0-49ed-bd1f-9cbdbf9e8a5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:19.453799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 45/87] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:19.459028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009898a9-41ae-4eac-8309-fa8c4acd1154 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:19.475701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc32858-ffe0-49ed-bd1f-9cbdbf9e8a5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:19.477772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc32858-ffe0-49ed-bd1f-9cbdbf9e8a5e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:19.510164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:19.554204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:19.566724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009898a9-41ae-4eac-8309-fa8c4acd1154 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:19.611073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ca160-2047-4d20-9392-a95a632f1dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:19.611968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ca160-2047-4d20-9392-a95a632f1dff None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:19.612254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ca160-2047-4d20-9392-a95a632f1dff None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:19.612565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ca160-2047-4d20-9392-a95a632f1dff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:19.614112 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 43/88] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:19.615073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc32858-ffe0-49ed-bd1f-9cbdbf9e8a5e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:19.712843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ca160-2047-4d20-9392-a95a632f1dff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:19.715441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ca160-2047-4d20-9392-a95a632f1dff None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:19.731228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc32858-ffe0-49ed-bd1f-9cbdbf9e8a5e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:19.771521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:19.775863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063517a-91a7-486d-b2d0-59974b9a65a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:19.776701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063517a-91a7-486d-b2d0-59974b9a65a1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:19.776815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063517a-91a7-486d-b2d0-59974b9a65a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:19.777078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063517a-91a7-486d-b2d0-59974b9a65a1 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:19.778898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063517a-91a7-486d-b2d0-59974b9a65a1 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:19.781002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063517a-91a7-486d-b2d0-59974b9a65a1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:19.783086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 44/89] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:19 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:19.783960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063517a-91a7-486d-b2d0-59974b9a65a1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:19.818106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:19.839308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 46/90] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:19.839308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ca160-2047-4d20-9392-a95a632f1dff None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:19.922151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063517a-91a7-486d-b2d0-59974b9a65a1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:19.965202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658ec03-449f-4461-995b-199af9fa5400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:19.966218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ec03-449f-4461-995b-199af9fa5400 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:19.966689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ec03-449f-4461-995b-199af9fa5400 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:19.967173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ec03-449f-4461-995b-199af9fa5400 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:19.968535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658ec03-449f-4461-995b-199af9fa5400 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:19.968642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ca160-2047-4d20-9392-a95a632f1dff None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:19.970292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658ec03-449f-4461-995b-199af9fa5400 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:19.972226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 45/91] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:19 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:19.974294 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658ec03-449f-4461-995b-199af9fa5400 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:20.013251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d5653-52e1-4b3f-9656-b01ab9d7b03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:20.013918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d5653-52e1-4b3f-9656-b01ab9d7b03c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:20.014516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d5653-52e1-4b3f-9656-b01ab9d7b03c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:20.014865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d5653-52e1-4b3f-9656-b01ab9d7b03c None admin] PATH_INFO: `/v3/projects/b5a5c662c30541dd9d9b9e1140540e19` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:20.016084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d5653-52e1-4b3f-9656-b01ab9d7b03c None admin] RBAC: Authorizing `identity:get_project(project_id=b5a5c662c30541dd9d9b9e1140540e19)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:20.018791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d5653-52e1-4b3f-9656-b01ab9d7b03c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:20.020847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 47/92] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:19 2022] GET /identity/v3/projects/b5a5c662c30541dd9d9b9e1140540e19 => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:20.022011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d5653-52e1-4b3f-9656-b01ab9d7b03c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:20.048625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:20.088705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:20.103775 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658ec03-449f-4461-995b-199af9fa5400 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:20.148844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d5653-52e1-4b3f-9656-b01ab9d7b03c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:20.149141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64f826-79b1-4c77-85ab-83c30bf0217c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:20.149735 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64f826-79b1-4c77-85ab-83c30bf0217c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:20.150015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64f826-79b1-4c77-85ab-83c30bf0217c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:20.150286 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64f826-79b1-4c77-85ab-83c30bf0217c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:20.151008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64f826-79b1-4c77-85ab-83c30bf0217c None admin] RBAC: Adding query filter params (role.id=c70d64f54bff4d2ba00749c556be9c49, scope.project.id=3d52ac8525814206ade19cf3bc6b9271, user.id=8e67b785954f4fd4b095552017407c73) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:20.151315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64f826-79b1-4c77-85ab-83c30bf0217c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:20.154419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64f826-79b1-4c77-85ab-83c30bf0217c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:20.165096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 46/93] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:19 2022] GET /identity/v3/role_assignments?user.id=8e67b785954f4fd4b095552017407c73&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:20.165922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64f826-79b1-4c77-85ab-83c30bf0217c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:20.192362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e738f-69b4-45df-be36-aa2d684b0310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:20.193587 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e738f-69b4-45df-be36-aa2d684b0310 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:20.194087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e738f-69b4-45df-be36-aa2d684b0310 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:20.194567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e738f-69b4-45df-be36-aa2d684b0310 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:20.196033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e738f-69b4-45df-be36-aa2d684b0310 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:20.197781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e738f-69b4-45df-be36-aa2d684b0310 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:20.200140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 48/94] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:20 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:20.206171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e738f-69b4-45df-be36-aa2d684b0310 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:20.361961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64f826-79b1-4c77-85ab-83c30bf0217c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:20.378849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:20.396934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e738f-69b4-45df-be36-aa2d684b0310 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:20.413585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebb4ca-30d1-46d1-b020-fdee80db0ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:20.414572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebb4ca-30d1-46d1-b020-fdee80db0ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:20.415047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebb4ca-30d1-46d1-b020-fdee80db0ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:20.415510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebb4ca-30d1-46d1-b020-fdee80db0ae3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:20.416424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebb4ca-30d1-46d1-b020-fdee80db0ae3 None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=3d52ac8525814206ade19cf3bc6b9271, user.id=824ec611f32e40609c56e94f7edd552e) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:20.416979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebb4ca-30d1-46d1-b020-fdee80db0ae3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:20.419490 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebb4ca-30d1-46d1-b020-fdee80db0ae3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:20.426849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 47/95] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:20 2022] GET /identity/v3/role_assignments?user.id=824ec611f32e40609c56e94f7edd552e&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:20.427954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebb4ca-30d1-46d1-b020-fdee80db0ae3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:20.452765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:20.483140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07663267-bb42-4b53-91f0-27735e7e299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:20.483826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07663267-bb42-4b53-91f0-27735e7e299c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:20.484118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07663267-bb42-4b53-91f0-27735e7e299c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:20.484401 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07663267-bb42-4b53-91f0-27735e7e299c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:20.485259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07663267-bb42-4b53-91f0-27735e7e299c None admin] RBAC: Adding query filter params (role.id=c70d64f54bff4d2ba00749c556be9c49, scope.project.id=b5a5c662c30541dd9d9b9e1140540e19, user.id=8e67b785954f4fd4b095552017407c73) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:20.485988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07663267-bb42-4b53-91f0-27735e7e299c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:20.487234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07663267-bb42-4b53-91f0-27735e7e299c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:20.504304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 49/96] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:20 2022] GET /identity/v3/role_assignments?user.id=8e67b785954f4fd4b095552017407c73&scope.project.id=b5a5c662c30541dd9d9b9e1140540e19&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:20.561233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebb4ca-30d1-46d1-b020-fdee80db0ae3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:20.602456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02015cb2-e229-4519-aecc-b6437c6fa7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:20.603471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02015cb2-e229-4519-aecc-b6437c6fa7fe None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:20.603951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02015cb2-e229-4519-aecc-b6437c6fa7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:20.604422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02015cb2-e229-4519-aecc-b6437c6fa7fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:20.606126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02015cb2-e229-4519-aecc-b6437c6fa7fe None admin] RBAC: Adding query filter params (role.id=6f9c4ad710e84eaabc415a1dd1541055, scope.project.id=3d52ac8525814206ade19cf3bc6b9271, user.id=8e67b785954f4fd4b095552017407c73) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:20.606665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02015cb2-e229-4519-aecc-b6437c6fa7fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:20.608524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02015cb2-e229-4519-aecc-b6437c6fa7fe None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:20.615429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 48/97] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:20 2022] GET /identity/v3/role_assignments?user.id=8e67b785954f4fd4b095552017407c73&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=6f9c4ad710e84eaabc415a1dd1541055 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:20.988411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182948d1-74e6-4d85-a515-2ef6abf28d48 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:20.988777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182948d1-74e6-4d85-a515-2ef6abf28d48 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:20.988968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182948d1-74e6-4d85-a515-2ef6abf28d48 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:20.990389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 50/98] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:20.995781 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4e68a-4c59-4a17-8650-ddca278002b1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:20.996628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4e68a-4c59-4a17-8650-ddca278002b1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:20.996910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4e68a-4c59-4a17-8650-ddca278002b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:21.021504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-14b4e68a-4c59-4a17-8650-ddca278002b1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:21.022711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-14b4e68a-4c59-4a17-8650-ddca278002b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:21.099021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84f1d8-2a05-43a4-9f03-0b729ba9e7db None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:21.099331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84f1d8-2a05-43a4-9f03-0b729ba9e7db None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:21.099851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84f1d8-2a05-43a4-9f03-0b729ba9e7db None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:21.101219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 51/99] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:29:21.106325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed3214-c638-48cb-a4ff-6e0c5dbda6e2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:21.106607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed3214-c638-48cb-a4ff-6e0c5dbda6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:21.106885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed3214-c638-48cb-a4ff-6e0c5dbda6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:21.129928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 49/100] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:21.134493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0eed3214-c638-48cb-a4ff-6e0c5dbda6e2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:21.135703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0eed3214-c638-48cb-a4ff-6e0c5dbda6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:21.219052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 52/101] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:21.544321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4e68a-4c59-4a17-8650-ddca278002b1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:21.559192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:21.672935 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4e68a-4c59-4a17-8650-ddca278002b1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:21.721185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5001c7-a5d1-4c6f-999e-74916015ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:21.721881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5001c7-a5d1-4c6f-999e-74916015ff1c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:21.722167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5001c7-a5d1-4c6f-999e-74916015ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:21.722438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5001c7-a5d1-4c6f-999e-74916015ff1c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:21.736187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5001c7-a5d1-4c6f-999e-74916015ff1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2b5001c7-a5d1-4c6f-999e-74916015ff1c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:21.744409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:21.745273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:21.746605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 50/102] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:21.750370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed3214-c638-48cb-a4ff-6e0c5dbda6e2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:21.767544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5001c7-a5d1-4c6f-999e-74916015ff1c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:21.871635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:21.896101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed3214-c638-48cb-a4ff-6e0c5dbda6e2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:21.938966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0c406-1f2f-4ca2-ada8-632d97b5a209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:21.939602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0c406-1f2f-4ca2-ada8-632d97b5a209 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:21.939883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0c406-1f2f-4ca2-ada8-632d97b5a209 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:21.940155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0c406-1f2f-4ca2-ada8-632d97b5a209 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:21.944269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0c406-1f2f-4ca2-ada8-632d97b5a209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:21.945768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0c406-1f2f-4ca2-ada8-632d97b5a209 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:21.947337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 53/103] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:21.954091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0c406-1f2f-4ca2-ada8-632d97b5a209 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:21.971223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:21.990395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5001c7-a5d1-4c6f-999e-74916015ff1c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:22.070400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce544766-b23e-47d2-b854-a340de558333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:22.071030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce544766-b23e-47d2-b854-a340de558333 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:22.071316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce544766-b23e-47d2-b854-a340de558333 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:22.071597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce544766-b23e-47d2-b854-a340de558333 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:22.072680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce544766-b23e-47d2-b854-a340de558333 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:22.072996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce544766-b23e-47d2-b854-a340de558333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:22.075672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce544766-b23e-47d2-b854-a340de558333 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:22.095481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 51/104] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:22.100750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce544766-b23e-47d2-b854-a340de558333 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:22.237284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0c406-1f2f-4ca2-ada8-632d97b5a209 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:22.317383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904b3ea-9d60-4c8c-85d8-8c8042ccdfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:22.318030 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904b3ea-9d60-4c8c-85d8-8c8042ccdfa5 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:22.318319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904b3ea-9d60-4c8c-85d8-8c8042ccdfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:22.318604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904b3ea-9d60-4c8c-85d8-8c8042ccdfa5 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:22.319411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904b3ea-9d60-4c8c-85d8-8c8042ccdfa5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:22.321037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904b3ea-9d60-4c8c-85d8-8c8042ccdfa5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:22.330539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce544766-b23e-47d2-b854-a340de558333 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:22.367373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:22.380319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 54/105] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:29:21 2022] POST /identity/v3/projects => generated 302 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:22.394738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecd4f7-c6b5-4ffa-8723-752039d63eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:22.395536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd4f7-c6b5-4ffa-8723-752039d63eef None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:22.395825 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd4f7-c6b5-4ffa-8723-752039d63eef None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:22.396099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd4f7-c6b5-4ffa-8723-752039d63eef None admin] PATH_INFO: `/v3/users/29822e5b266140fab42679a3fe2533fb` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:22.403127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:22.425449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecd4f7-c6b5-4ffa-8723-752039d63eef None admin] RBAC: Authorizing `identity:get_user(user_id=29822e5b266140fab42679a3fe2533fb)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:22.427021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecd4f7-c6b5-4ffa-8723-752039d63eef None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:22.429124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 52/106] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:22 2022] GET /identity/v3/users/29822e5b266140fab42679a3fe2533fb => generated 289 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:22.435927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904b3ea-9d60-4c8c-85d8-8c8042ccdfa5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:22.616750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904b3ea-9d60-4c8c-85d8-8c8042ccdfa5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:22.660862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c7f03-fb4a-49e7-8b3d-bb6f426fbee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:22.661485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c7f03-fb4a-49e7-8b3d-bb6f426fbee9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:22.661774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c7f03-fb4a-49e7-8b3d-bb6f426fbee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:22.662052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c7f03-fb4a-49e7-8b3d-bb6f426fbee9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:22.742186 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:22.746530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c7f03-fb4a-49e7-8b3d-bb6f426fbee9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:22.749015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c7f03-fb4a-49e7-8b3d-bb6f426fbee9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:22.753917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef31ad2-f0e5-43ff-8fb0-1e1dbfa4fcdf None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:22.756919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef31ad2-f0e5-43ff-8fb0-1e1dbfa4fcdf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:22.757494 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef31ad2-f0e5-43ff-8fb0-1e1dbfa4fcdf None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:22.759071 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 53/107] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:22.769629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9f303-dcb4-45d9-8fda-305c73315634 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:22.770679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9f303-dcb4-45d9-8fda-305c73315634 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:22.771164 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9f303-dcb4-45d9-8fda-305c73315634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:22.812852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a1d9f303-dcb4-45d9-8fda-305c73315634 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:22.814157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a1d9f303-dcb4-45d9-8fda-305c73315634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:22.950566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 55/108] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:22.957546 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c7f03-fb4a-49e7-8b3d-bb6f426fbee9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:22.962871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 54/109] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:23.089897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:23.094856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c7f03-fb4a-49e7-8b3d-bb6f426fbee9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:23.141924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25636a7e-3de0-4c97-944e-0a93d8635da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:23.142830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25636a7e-3de0-4c97-944e-0a93d8635da6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:23.143273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25636a7e-3de0-4c97-944e-0a93d8635da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:23.143705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25636a7e-3de0-4c97-944e-0a93d8635da6 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:23.149203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25636a7e-3de0-4c97-944e-0a93d8635da6 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:23.150827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25636a7e-3de0-4c97-944e-0a93d8635da6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:23.153301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 56/110] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:22 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:23.159408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9f303-dcb4-45d9-8fda-305c73315634 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:23.286854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9f303-dcb4-45d9-8fda-305c73315634 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:23.310992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:23.327546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3277000-b176-468a-8642-3d0534d97639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:23.328275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3277000-b176-468a-8642-3d0534d97639 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:23.328578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3277000-b176-468a-8642-3d0534d97639 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:23.328854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3277000-b176-468a-8642-3d0534d97639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:23.329582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3277000-b176-468a-8642-3d0534d97639 None admin] RBAC: Adding query filter params (role.id=4febe0fde4a74359af9325b7352b75d6, scope.project.id=3d52ac8525814206ade19cf3bc6b9271, user.id=29822e5b266140fab42679a3fe2533fb) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:23.329888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3277000-b176-468a-8642-3d0534d97639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:23.331415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3277000-b176-468a-8642-3d0534d97639 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:23.337074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 55/111] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:23 2022] GET /identity/v3/role_assignments?user.id=29822e5b266140fab42679a3fe2533fb&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=4febe0fde4a74359af9325b7352b75d6 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:23.398818 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25636a7e-3de0-4c97-944e-0a93d8635da6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:23.464875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cc3fb-c4bd-4fb4-9439-33158348a061 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:23.465290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cc3fb-c4bd-4fb4-9439-33158348a061 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:23.465635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cc3fb-c4bd-4fb4-9439-33158348a061 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:23.467242 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 56/112] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:23.502838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8304bc4-0f7e-4508-98e4-55ce54f6c6e8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:23.503177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8304bc4-0f7e-4508-98e4-55ce54f6c6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:23.503394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8304bc4-0f7e-4508-98e4-55ce54f6c6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:23.536308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25636a7e-3de0-4c97-944e-0a93d8635da6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:23.548609 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c8304bc4-0f7e-4508-98e4-55ce54f6c6e8 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:23.549708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c8304bc4-0f7e-4508-98e4-55ce54f6c6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:23.591942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede2826-ebdc-40f9-89c1-00ddd178de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:23.592682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede2826-ebdc-40f9-89c1-00ddd178de42 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:23.592973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede2826-ebdc-40f9-89c1-00ddd178de42 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:23.593257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede2826-ebdc-40f9-89c1-00ddd178de42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:23.614515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede2826-ebdc-40f9-89c1-00ddd178de42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-5ede2826-ebdc-40f9-89c1-00ddd178de42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:23.627840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:23.628775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:23.629549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 57/113] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:23.636760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede2826-ebdc-40f9-89c1-00ddd178de42 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:23.679369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:23.696938 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 57/114] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:23.698509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f20a06-dafe-486d-a84c-95221d5f171c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:23.701534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f20a06-dafe-486d-a84c-95221d5f171c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:23.702011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f20a06-dafe-486d-a84c-95221d5f171c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:23.703535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 58/115] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:23.709311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e2ba5-2bea-425b-9cc2-a9251f9a9ed2 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:23.709794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e2ba5-2bea-425b-9cc2-a9251f9a9ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:23.710279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e2ba5-2bea-425b-9cc2-a9251f9a9ed2 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:23.712173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 59/116] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:23.713441 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924b21-9fbc-4fd2-b80a-0558390fb4bd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:23.713944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924b21-9fbc-4fd2-b80a-0558390fb4bd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:23.714427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924b21-9fbc-4fd2-b80a-0558390fb4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:23.752628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-91924b21-9fbc-4fd2-b80a-0558390fb4bd None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:23.753824 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-91924b21-9fbc-4fd2-b80a-0558390fb4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:23.816576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede2826-ebdc-40f9-89c1-00ddd178de42 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:23.886215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 60/117] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:23.887565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3cc49-7133-41c7-8cac-a5515760242e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:23.887871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3cc49-7133-41c7-8cac-a5515760242e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:23.888706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3cc49-7133-41c7-8cac-a5515760242e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:23.895207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57391a6b-ae8c-43d4-a770-893701b6b609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:23.895825 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57391a6b-ae8c-43d4-a770-893701b6b609 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:23.896109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57391a6b-ae8c-43d4-a770-893701b6b609 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:23.896385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57391a6b-ae8c-43d4-a770-893701b6b609 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:23.897092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57391a6b-ae8c-43d4-a770-893701b6b609 None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:23.897393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57391a6b-ae8c-43d4-a770-893701b6b609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:23.898777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57391a6b-ae8c-43d4-a770-893701b6b609 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:23.922741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 58/118] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:23.923692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-24f3cc49-7133-41c7-8cac-a5515760242e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:23.924974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-24f3cc49-7133-41c7-8cac-a5515760242e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:23.929681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57391a6b-ae8c-43d4-a770-893701b6b609 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:24.016890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:24.056235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 61/119] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:24.056988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3cc49-7133-41c7-8cac-a5515760242e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:24.083773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:24.111093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57391a6b-ae8c-43d4-a770-893701b6b609 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:24.185286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c94d4-b0c8-4fdd-b257-fb675b6e2067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:24.185952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c94d4-b0c8-4fdd-b257-fb675b6e2067 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:24.186240 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c94d4-b0c8-4fdd-b257-fb675b6e2067 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:24.186522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c94d4-b0c8-4fdd-b257-fb675b6e2067 None admin] PATH_INFO: `/v3/users/8e67b785954f4fd4b095552017407c73` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:24.193181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c94d4-b0c8-4fdd-b257-fb675b6e2067 None admin] RBAC: Authorizing `identity:get_user(user_id=8e67b785954f4fd4b095552017407c73)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:24.194650 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c94d4-b0c8-4fdd-b257-fb675b6e2067 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:24.196426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 59/120] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:23 2022] GET /identity/v3/users/8e67b785954f4fd4b095552017407c73 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:24.209494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c94d4-b0c8-4fdd-b257-fb675b6e2067 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:24.216352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3cc49-7133-41c7-8cac-a5515760242e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:24.273312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d0611-77a3-4aab-82f0-64ee34f20276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:24.273961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d0611-77a3-4aab-82f0-64ee34f20276 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:24.274243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d0611-77a3-4aab-82f0-64ee34f20276 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:24.274512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d0611-77a3-4aab-82f0-64ee34f20276 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:24.289523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d0611-77a3-4aab-82f0-64ee34f20276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2e9d0611-77a3-4aab-82f0-64ee34f20276 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:24.295297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:24.297374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:24.299251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:24.304862 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 62/121] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:29:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:24.305218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d0611-77a3-4aab-82f0-64ee34f20276 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:24.360534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c94d4-b0c8-4fdd-b257-fb675b6e2067 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:24.389913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:24.405830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdc892-4f27-47da-bf55-1fc8cad0c9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:24.406483 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc892-4f27-47da-bf55-1fc8cad0c9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:24.406772 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc892-4f27-47da-bf55-1fc8cad0c9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:24.407054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc892-4f27-47da-bf55-1fc8cad0c9d8 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:24.408335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdc892-4f27-47da-bf55-1fc8cad0c9d8 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:24.409844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdc892-4f27-47da-bf55-1fc8cad0c9d8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:24.414916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 60/122] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:24 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:24.415926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdc892-4f27-47da-bf55-1fc8cad0c9d8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:24.455933 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:24.457931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d0611-77a3-4aab-82f0-64ee34f20276 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:24.507914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf55ca-6e42-4747-932c-71e51b284f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:24.509194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf55ca-6e42-4747-932c-71e51b284f3a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:24.509482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf55ca-6e42-4747-932c-71e51b284f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:24.509759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf55ca-6e42-4747-932c-71e51b284f3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:24.524569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf55ca-6e42-4747-932c-71e51b284f3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8acf55ca-6e42-4747-932c-71e51b284f3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:24.531546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:24.532407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:24.533280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:24.533850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 63/123] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:24.534814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf55ca-6e42-4747-932c-71e51b284f3a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:24.557695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdc892-4f27-47da-bf55-1fc8cad0c9d8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:24.598697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236dbad-fe80-4469-8011-5ed328842f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:24.599660 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236dbad-fe80-4469-8011-5ed328842f46 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:24.600149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236dbad-fe80-4469-8011-5ed328842f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:24.600769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236dbad-fe80-4469-8011-5ed328842f46 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:24.601743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236dbad-fe80-4469-8011-5ed328842f46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:24.602344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236dbad-fe80-4469-8011-5ed328842f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:24.604078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236dbad-fe80-4469-8011-5ed328842f46 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:24.614839 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 61/124] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:29:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:24.615895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236dbad-fe80-4469-8011-5ed328842f46 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:24.642643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:24.648776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf55ca-6e42-4747-932c-71e51b284f3a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:24.696119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b2ea2-47d3-46a1-8b4d-90db3fcc7f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:24.696789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2ea2-47d3-46a1-8b4d-90db3fcc7f5d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:24.697076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2ea2-47d3-46a1-8b4d-90db3fcc7f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:24.697355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2ea2-47d3-46a1-8b4d-90db3fcc7f5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:24.710642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b2ea2-47d3-46a1-8b4d-90db3fcc7f5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-025b2ea2-47d3-46a1-8b4d-90db3fcc7f5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:24.717636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:24.718498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:24.719257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 64/125] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:29:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:24.719937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b2ea2-47d3-46a1-8b4d-90db3fcc7f5d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:24.724687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:24.749335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236dbad-fe80-4469-8011-5ed328842f46 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:24.791851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98762d2e-0cfd-4f3e-a9df-d36c8668076c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:24.793495 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762d2e-0cfd-4f3e-a9df-d36c8668076c None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:24.794031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762d2e-0cfd-4f3e-a9df-d36c8668076c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:24.794517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762d2e-0cfd-4f3e-a9df-d36c8668076c None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271/users/8e67b785954f4fd4b095552017407c73/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:24.796785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98762d2e-0cfd-4f3e-a9df-d36c8668076c None admin] RBAC: Authorizing `identity:create_grant(project_id=3d52ac8525814206ade19cf3bc6b9271, user_id=8e67b785954f4fd4b095552017407c73, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:24.798497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98762d2e-0cfd-4f3e-a9df-d36c8668076c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:24.814219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 62/126] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:29:24 2022] PUT /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271/users/8e67b785954f4fd4b095552017407c73/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:24.815450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98762d2e-0cfd-4f3e-a9df-d36c8668076c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:24.894177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b2ea2-47d3-46a1-8b4d-90db3fcc7f5d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:24.931284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98762d2e-0cfd-4f3e-a9df-d36c8668076c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:24.959847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88ad03-ca06-44a5-ae41-e2e01201da7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:24.960532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88ad03-ca06-44a5-ae41-e2e01201da7b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:24.960807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88ad03-ca06-44a5-ae41-e2e01201da7b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:24.961049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88ad03-ca06-44a5-ae41-e2e01201da7b None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:24.961727 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88ad03-ca06-44a5-ae41-e2e01201da7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:24.962032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88ad03-ca06-44a5-ae41-e2e01201da7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:24.963549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88ad03-ca06-44a5-ae41-e2e01201da7b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:24.972689 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0085839-b45b-4402-9572-d47dea316e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:24.973634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0085839-b45b-4402-9572-d47dea316e36 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:24.974185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0085839-b45b-4402-9572-d47dea316e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:24.974657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0085839-b45b-4402-9572-d47dea316e36 None admin] PATH_INFO: `/v3/users/8e67b785954f4fd4b095552017407c73` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:24.976204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0085839-b45b-4402-9572-d47dea316e36 None admin] RBAC: Authorizing `identity:get_user(user_id=8e67b785954f4fd4b095552017407c73)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:24.977942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0085839-b45b-4402-9572-d47dea316e36 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:24.980159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 63/127] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:24 2022] GET /identity/v3/users/8e67b785954f4fd4b095552017407c73 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:24.980683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 65/128] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:24.981901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0085839-b45b-4402-9572-d47dea316e36 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:24.986648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88ad03-ca06-44a5-ae41-e2e01201da7b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:25.006508 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:25.043913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:25.099762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0085839-b45b-4402-9572-d47dea316e36 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:25.147666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b26ab-393d-4224-9340-6dc9b40661d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:25.148541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b26ab-393d-4224-9340-6dc9b40661d0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.148842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b26ab-393d-4224-9340-6dc9b40661d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.149129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b26ab-393d-4224-9340-6dc9b40661d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.150090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b26ab-393d-4224-9340-6dc9b40661d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:25.150408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b26ab-393d-4224-9340-6dc9b40661d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:25.151797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b26ab-393d-4224-9340-6dc9b40661d0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:25.162953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 64/129] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:25.163802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b26ab-393d-4224-9340-6dc9b40661d0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:25.205228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88ad03-ca06-44a5-ae41-e2e01201da7b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:25.251286 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701304d-7e20-4505-a704-839a2dd853f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:25.251980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701304d-7e20-4505-a704-839a2dd853f4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.252273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701304d-7e20-4505-a704-839a2dd853f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.252578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701304d-7e20-4505-a704-839a2dd853f4 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.254209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701304d-7e20-4505-a704-839a2dd853f4 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:25.255638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701304d-7e20-4505-a704-839a2dd853f4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:25.255959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:25.257729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 66/130] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:24 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:25.258636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701304d-7e20-4505-a704-839a2dd853f4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:25.310886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b26ab-393d-4224-9340-6dc9b40661d0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:25.364255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:25.382561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701304d-7e20-4505-a704-839a2dd853f4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:25.384023 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8fdae-242e-4eb1-bed7-f8c65c76b7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:25.384804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8fdae-242e-4eb1-bed7-f8c65c76b7bb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.385195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8fdae-242e-4eb1-bed7-f8c65c76b7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.386215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8fdae-242e-4eb1-bed7-f8c65c76b7bb None admin] PATH_INFO: `/v3/users/8e67b785954f4fd4b095552017407c73` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.389274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8fdae-242e-4eb1-bed7-f8c65c76b7bb None admin] RBAC: Authorizing `identity:get_user(user_id=8e67b785954f4fd4b095552017407c73)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:25.390791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8fdae-242e-4eb1-bed7-f8c65c76b7bb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:25.392820 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 65/131] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:25 2022] GET /identity/v3/users/8e67b785954f4fd4b095552017407c73 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:25.397798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8fdae-242e-4eb1-bed7-f8c65c76b7bb None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:25.423834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b99597-40a4-4dbe-b40c-b6092a6e4a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:25.424750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b99597-40a4-4dbe-b40c-b6092a6e4a55 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.425425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b99597-40a4-4dbe-b40c-b6092a6e4a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.425904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b99597-40a4-4dbe-b40c-b6092a6e4a55 None admin] PATH_INFO: `/v3/users/824ec611f32e40609c56e94f7edd552e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.427339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b99597-40a4-4dbe-b40c-b6092a6e4a55 None admin] RBAC: Authorizing `identity:get_user(user_id=824ec611f32e40609c56e94f7edd552e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:25.428972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b99597-40a4-4dbe-b40c-b6092a6e4a55 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:25.431104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 67/132] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:25 2022] GET /identity/v3/users/824ec611f32e40609c56e94f7edd552e => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:25.432144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b99597-40a4-4dbe-b40c-b6092a6e4a55 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:25.512323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8fdae-242e-4eb1-bed7-f8c65c76b7bb None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:25.550526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adcb44-78d7-478f-a789-c3a2039ff400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:25.551198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b99597-40a4-4dbe-b40c-b6092a6e4a55 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:25.551591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adcb44-78d7-478f-a789-c3a2039ff400 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.552046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adcb44-78d7-478f-a789-c3a2039ff400 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.552481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adcb44-78d7-478f-a789-c3a2039ff400 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271/users/8e67b785954f4fd4b095552017407c73/roles/6f9c4ad710e84eaabc415a1dd1541055` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.554920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55adcb44-78d7-478f-a789-c3a2039ff400 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d52ac8525814206ade19cf3bc6b9271, user_id=8e67b785954f4fd4b095552017407c73, role_id=6f9c4ad710e84eaabc415a1dd1541055)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:25.556696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55adcb44-78d7-478f-a789-c3a2039ff400 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:25.575488 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 66/133] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:29:25 2022] PUT /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271/users/8e67b785954f4fd4b095552017407c73/roles/6f9c4ad710e84eaabc415a1dd1541055 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:25.577776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935c741-2ce6-4dbb-8251-44614d0ce250 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.578620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935c741-2ce6-4dbb-8251-44614d0ce250 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.579101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935c741-2ce6-4dbb-8251-44614d0ce250 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.581254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 67/134] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:25.582241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935c741-2ce6-4dbb-8251-44614d0ce250 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:25.600184 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:25.613085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b32d4b-2cc8-49b0-bd5e-78af60fae08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:25.613752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32d4b-2cc8-49b0-bd5e-78af60fae08d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.614038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32d4b-2cc8-49b0-bd5e-78af60fae08d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.614316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32d4b-2cc8-49b0-bd5e-78af60fae08d None admin] PATH_INFO: `/v3/projects/b5a5c662c30541dd9d9b9e1140540e19` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.616782 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b32d4b-2cc8-49b0-bd5e-78af60fae08d None admin] RBAC: Authorizing `identity:get_project(project_id=b5a5c662c30541dd9d9b9e1140540e19)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:25.618280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b32d4b-2cc8-49b0-bd5e-78af60fae08d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:25.620082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 68/135] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:25 2022] GET /identity/v3/projects/b5a5c662c30541dd9d9b9e1140540e19 => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:25.623576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b1ebc-7184-4bf5-8cd3-265658c69097 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.623868 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b1ebc-7184-4bf5-8cd3-265658c69097 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.624152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b1ebc-7184-4bf5-8cd3-265658c69097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.664316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3a3b1ebc-7184-4bf5-8cd3-265658c69097 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:25.665539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3a3b1ebc-7184-4bf5-8cd3-265658c69097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:25.707692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:25.773782 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 69/136] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:25.774627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b1ebc-7184-4bf5-8cd3-265658c69097 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:25.774749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935c741-2ce6-4dbb-8251-44614d0ce250 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:25.819485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e056e4-fe2a-4b77-b57e-105dddd1d639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:25.820126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e056e4-fe2a-4b77-b57e-105dddd1d639 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.820414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e056e4-fe2a-4b77-b57e-105dddd1d639 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.820741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e056e4-fe2a-4b77-b57e-105dddd1d639 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.822076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e056e4-fe2a-4b77-b57e-105dddd1d639 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:25.823336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e056e4-fe2a-4b77-b57e-105dddd1d639 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:25.825562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 68/137] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:25 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:25.826907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40748a37-b67d-4041-b2c7-8ea929ae5852 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.827196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40748a37-b67d-4041-b2c7-8ea929ae5852 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.827474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40748a37-b67d-4041-b2c7-8ea929ae5852 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.832892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 69/138] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:25.833690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40748a37-b67d-4041-b2c7-8ea929ae5852 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:25.903431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b1ebc-7184-4bf5-8cd3-265658c69097 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:25.929526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:25.943561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a2259-7879-44e4-8ad5-b6625cddf2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:25.944607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a2259-7879-44e4-8ad5-b6625cddf2a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.945084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a2259-7879-44e4-8ad5-b6625cddf2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.945553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a2259-7879-44e4-8ad5-b6625cddf2a2 None admin] PATH_INFO: `/v3/projects/b5a5c662c30541dd9d9b9e1140540e19/users/8e67b785954f4fd4b095552017407c73/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.947883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a2259-7879-44e4-8ad5-b6625cddf2a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5a5c662c30541dd9d9b9e1140540e19, user_id=8e67b785954f4fd4b095552017407c73, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:25.950423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a2259-7879-44e4-8ad5-b6625cddf2a2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:25.972186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 70/139] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:29:25 2022] PUT /identity/v3/projects/b5a5c662c30541dd9d9b9e1140540e19/users/8e67b785954f4fd4b095552017407c73/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:25.974899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ea171-a8ad-4caa-84f7-cdf782fc5fa2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:25.975990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ea171-a8ad-4caa-84f7-cdf782fc5fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:25.976828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ea171-a8ad-4caa-84f7-cdf782fc5fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:25.998475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40748a37-b67d-4041-b2c7-8ea929ae5852 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:26.015952 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-014ea171-a8ad-4caa-84f7-cdf782fc5fa2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:26.017680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-014ea171-a8ad-4caa-84f7-cdf782fc5fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:26.026685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:26.070607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f51e3-a5d0-446f-b277-84999045d9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:26.070995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f51e3-a5d0-446f-b277-84999045d9b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:26.071276 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f51e3-a5d0-446f-b277-84999045d9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:26.071558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f51e3-a5d0-446f-b277-84999045d9b6 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271/users/824ec611f32e40609c56e94f7edd552e/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:26.078405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f51e3-a5d0-446f-b277-84999045d9b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d52ac8525814206ade19cf3bc6b9271, user_id=824ec611f32e40609c56e94f7edd552e, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:26.079591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f51e3-a5d0-446f-b277-84999045d9b6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:26.103572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 71/141] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:26.104781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 70/141] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:29:25 2022] PUT /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271/users/824ec611f32e40609c56e94f7edd552e/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:26.178322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ea171-a8ad-4caa-84f7-cdf782fc5fa2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:26.305512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ea171-a8ad-4caa-84f7-cdf782fc5fa2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:26.346701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff78971-0952-4993-8809-40a82e5ba97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:26.347671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff78971-0952-4993-8809-40a82e5ba97b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:26.348159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff78971-0952-4993-8809-40a82e5ba97b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:26.348680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff78971-0952-4993-8809-40a82e5ba97b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:26.349999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff78971-0952-4993-8809-40a82e5ba97b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:26.351676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff78971-0952-4993-8809-40a82e5ba97b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:26.353836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 72/142] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:26.362156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f51e3-a5d0-446f-b277-84999045d9b6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:26.495290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f51e3-a5d0-446f-b277-84999045d9b6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:26.502089 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff78971-0952-4993-8809-40a82e5ba97b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:26.530331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:26.540986 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:26.548330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226042f-c568-4e5e-b433-ca6c9b7b22b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:26.549203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226042f-c568-4e5e-b433-ca6c9b7b22b5 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:26.549631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226042f-c568-4e5e-b433-ca6c9b7b22b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:26.550046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226042f-c568-4e5e-b433-ca6c9b7b22b5 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:26.550924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226042f-c568-4e5e-b433-ca6c9b7b22b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:26.552549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226042f-c568-4e5e-b433-ca6c9b7b22b5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:26.585296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 71/143] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:29:26 2022] POST /identity/v3/users => generated 283 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:26.695440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff78971-0952-4993-8809-40a82e5ba97b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:26.750100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decfb29-4d5c-4ba8-bafe-959886db2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:26.750766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decfb29-4d5c-4ba8-bafe-959886db2fbe None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:26.751053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decfb29-4d5c-4ba8-bafe-959886db2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:26.751330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decfb29-4d5c-4ba8-bafe-959886db2fbe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:26.768085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2decfb29-4d5c-4ba8-bafe-959886db2fbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2decfb29-4d5c-4ba8-bafe-959886db2fbe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:26.773503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:26.774349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:26.775134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 73/144] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:29:26.782908 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226042f-c568-4e5e-b433-ca6c9b7b22b5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:26.940342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226042f-c568-4e5e-b433-ca6c9b7b22b5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:26.980424 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b162a7-af88-471e-9b5d-4532c51bd2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:26.981076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b162a7-af88-471e-9b5d-4532c51bd2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:26.981363 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b162a7-af88-471e-9b5d-4532c51bd2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:26.981640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b162a7-af88-471e-9b5d-4532c51bd2f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:26.982328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b162a7-af88-471e-9b5d-4532c51bd2f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:26.982638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b162a7-af88-471e-9b5d-4532c51bd2f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:26.984057 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b162a7-af88-471e-9b5d-4532c51bd2f0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:26.993639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 72/145] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:27.000885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decfb29-4d5c-4ba8-bafe-959886db2fbe None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:27.115334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:27.156793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decfb29-4d5c-4ba8-bafe-959886db2fbe None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:27.198105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215144ac-7583-4bc5-a368-417a20a3c959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:27.198730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215144ac-7583-4bc5-a368-417a20a3c959 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:27.199624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215144ac-7583-4bc5-a368-417a20a3c959 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:27.199901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215144ac-7583-4bc5-a368-417a20a3c959 None admin] PATH_INFO: `/v3/users/29822e5b266140fab42679a3fe2533fb` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:27.205237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215144ac-7583-4bc5-a368-417a20a3c959 None admin] RBAC: Authorizing `identity:get_user(user_id=29822e5b266140fab42679a3fe2533fb)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:27.206723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215144ac-7583-4bc5-a368-417a20a3c959 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:27.208689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 74/146] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:26 2022] GET /identity/v3/users/29822e5b266140fab42679a3fe2533fb => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:27.214234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b162a7-af88-471e-9b5d-4532c51bd2f0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:27.276125 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:27.340674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b162a7-af88-471e-9b5d-4532c51bd2f0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:27.382345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11147afa-8a3a-4439-bed4-21260dc09843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:27.382989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11147afa-8a3a-4439-bed4-21260dc09843 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:27.383324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11147afa-8a3a-4439-bed4-21260dc09843 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:27.383613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11147afa-8a3a-4439-bed4-21260dc09843 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:27.385560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11147afa-8a3a-4439-bed4-21260dc09843 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:27.387586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11147afa-8a3a-4439-bed4-21260dc09843 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:27.389257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 73/147] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:27 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:27.395143 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215144ac-7583-4bc5-a368-417a20a3c959 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:27.515333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215144ac-7583-4bc5-a368-417a20a3c959 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:27.555295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87c426-e3fd-482e-be3a-ad247d414b14 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:27.555625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87c426-e3fd-482e-be3a-ad247d414b14 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:27.555854 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87c426-e3fd-482e-be3a-ad247d414b14 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:27.557541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 74/148] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:27.559911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaaa49-d912-4c6c-950f-6b739ec7be0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:27.560733 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaaa49-d912-4c6c-950f-6b739ec7be0d None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:27.561115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaaa49-d912-4c6c-950f-6b739ec7be0d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:27.561491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaaa49-d912-4c6c-950f-6b739ec7be0d None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271/users/29822e5b266140fab42679a3fe2533fb/roles/4febe0fde4a74359af9325b7352b75d6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:27.564187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffb361-38a8-44e3-a9df-7111530385c0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:27.564473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffb361-38a8-44e3-a9df-7111530385c0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:27.564811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffb361-38a8-44e3-a9df-7111530385c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:27.570451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaaa49-d912-4c6c-950f-6b739ec7be0d None admin] RBAC: Authorizing `identity:create_grant(project_id=3d52ac8525814206ade19cf3bc6b9271, user_id=29822e5b266140fab42679a3fe2533fb, role_id=4febe0fde4a74359af9325b7352b75d6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:27.571917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaaa49-d912-4c6c-950f-6b739ec7be0d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:27.589380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-52ffb361-38a8-44e3-a9df-7111530385c0 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:27.590670 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-52ffb361-38a8-44e3-a9df-7111530385c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:27.593737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:27.599619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 75/149] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:29:27 2022] PUT /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271/users/29822e5b266140fab42679a3fe2533fb/roles/4febe0fde4a74359af9325b7352b75d6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:27.694472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 75/150] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:28.001459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaaa49-d912-4c6c-950f-6b739ec7be0d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:28.132301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaaa49-d912-4c6c-950f-6b739ec7be0d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:28.142390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9d7f7-1818-4eec-97c3-75b46b9404d3 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:28.142674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9d7f7-1818-4eec-97c3-75b46b9404d3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:28.142945 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9d7f7-1818-4eec-97c3-75b46b9404d3 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:28.144389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 76/151] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:28.150352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8115799-8bef-47c5-8098-809ff4e30633 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:28.150635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8115799-8bef-47c5-8098-809ff4e30633 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:28.150913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8115799-8bef-47c5-8098-809ff4e30633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:28.181536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d8115799-8bef-47c5-8098-809ff4e30633 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:28.182747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d8115799-8bef-47c5-8098-809ff4e30633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:28.202541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc244c02-be4b-4850-bb96-68014cdc083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:28.203183 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc244c02-be4b-4850-bb96-68014cdc083c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:28.203470 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc244c02-be4b-4850-bb96-68014cdc083c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:28.203739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc244c02-be4b-4850-bb96-68014cdc083c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:28.228787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc244c02-be4b-4850-bb96-68014cdc083c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-bc244c02-be4b-4850-bb96-68014cdc083c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:28.236053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:28.236939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:28.237757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:28.239495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 76/152] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:28.261997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc244c02-be4b-4850-bb96-68014cdc083c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:28.271368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:28.326141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 77/153] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:28.370108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:28.428030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc244c02-be4b-4850-bb96-68014cdc083c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:28.468083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2b720-194c-4857-8a82-63e66f7a0204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:28.468726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2b720-194c-4857-8a82-63e66f7a0204 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:28.469011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2b720-194c-4857-8a82-63e66f7a0204 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:28.469281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2b720-194c-4857-8a82-63e66f7a0204 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:28.469964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2b720-194c-4857-8a82-63e66f7a0204 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:28.470273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2b720-194c-4857-8a82-63e66f7a0204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:28.471675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2b720-194c-4857-8a82-63e66f7a0204 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:28.483902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 77/154] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:28.489323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8115799-8bef-47c5-8098-809ff4e30633 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:28.622680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8115799-8bef-47c5-8098-809ff4e30633 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:28.662584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11afd8f-c27c-4965-8dcc-622b2455384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:28.663219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11afd8f-c27c-4965-8dcc-622b2455384c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:28.663496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11afd8f-c27c-4965-8dcc-622b2455384c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:28.663764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11afd8f-c27c-4965-8dcc-622b2455384c None admin] PATH_INFO: `/v3/users/8e67b785954f4fd4b095552017407c73` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:28.664926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11afd8f-c27c-4965-8dcc-622b2455384c None admin] RBAC: Authorizing `identity:get_user(user_id=8e67b785954f4fd4b095552017407c73)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:28.666371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11afd8f-c27c-4965-8dcc-622b2455384c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:28.668250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 78/155] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:28 2022] GET /identity/v3/users/8e67b785954f4fd4b095552017407c73 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:28.683919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2b720-194c-4857-8a82-63e66f7a0204 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:28.808192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2b720-194c-4857-8a82-63e66f7a0204 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:28.818227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11afd8f-c27c-4965-8dcc-622b2455384c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:28.837586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:28.861525 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f5351-c60a-4470-b721-f7343d2362f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:28.862193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f5351-c60a-4470-b721-f7343d2362f9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:28.862473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f5351-c60a-4470-b721-f7343d2362f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:28.862745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f5351-c60a-4470-b721-f7343d2362f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:28.873088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:28.937582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11afd8f-c27c-4965-8dcc-622b2455384c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:28.958757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f5351-c60a-4470-b721-f7343d2362f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:28.960571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f5351-c60a-4470-b721-f7343d2362f9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:28.988279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f97043-b958-4b36-b36b-8c0d0ee4c66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:28.988998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f97043-b958-4b36-b36b-8c0d0ee4c66d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:28.989276 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f97043-b958-4b36-b36b-8c0d0ee4c66d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:28.989544 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f97043-b958-4b36-b36b-8c0d0ee4c66d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:29.001599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f97043-b958-4b36-b36b-8c0d0ee4c66d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-13f97043-b958-4b36-b36b-8c0d0ee4c66d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:29.007674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:29.008712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:29.009574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 79/156] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:29:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:29:29.025201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f97043-b958-4b36-b36b-8c0d0ee4c66d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:29.139136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 78/157] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:29.147774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f5351-c60a-4470-b721-f7343d2362f9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:29.187441 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:29.206016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f97043-b958-4b36-b36b-8c0d0ee4c66d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:29.215053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:29.272270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cf1ca-a897-4c89-9feb-413d4e83d90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:29.272947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cf1ca-a897-4c89-9feb-413d4e83d90e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:29.273189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cf1ca-a897-4c89-9feb-413d4e83d90e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:29.273488 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cf1ca-a897-4c89-9feb-413d4e83d90e None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:29.274176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cf1ca-a897-4c89-9feb-413d4e83d90e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:29.274485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cf1ca-a897-4c89-9feb-413d4e83d90e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:29.276110 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cf1ca-a897-4c89-9feb-413d4e83d90e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:29.293224 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 80/158] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:29:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:29.298984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cf1ca-a897-4c89-9feb-413d4e83d90e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:29.362663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f5351-c60a-4470-b721-f7343d2362f9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:29.466515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356902f1-1073-429d-8242-3bfc81bba61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:29.467075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356902f1-1073-429d-8242-3bfc81bba61a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:29.467352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356902f1-1073-429d-8242-3bfc81bba61a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:29.467619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356902f1-1073-429d-8242-3bfc81bba61a None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:29.473198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356902f1-1073-429d-8242-3bfc81bba61a None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:29.474664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356902f1-1073-429d-8242-3bfc81bba61a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:29.476333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 79/159] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:29 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:29.491336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356902f1-1073-429d-8242-3bfc81bba61a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:29.529435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cf1ca-a897-4c89-9feb-413d4e83d90e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:29.625548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece53985-4564-4ced-aaa2-31eb7a2a8528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:29.626612 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece53985-4564-4ced-aaa2-31eb7a2a8528 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:29.627112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece53985-4564-4ced-aaa2-31eb7a2a8528 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:29.627403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece53985-4564-4ced-aaa2-31eb7a2a8528 None admin] PATH_INFO: `/v3/users/8e67b785954f4fd4b095552017407c73` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:29.629095 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece53985-4564-4ced-aaa2-31eb7a2a8528 None admin] RBAC: Authorizing `identity:get_user(user_id=8e67b785954f4fd4b095552017407c73)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:29.630767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece53985-4564-4ced-aaa2-31eb7a2a8528 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:29.632727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 81/160] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:29 2022] GET /identity/v3/users/8e67b785954f4fd4b095552017407c73 => generated 275 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:29.638994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece53985-4564-4ced-aaa2-31eb7a2a8528 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:29.721072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:29.738878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:29.755375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356902f1-1073-429d-8242-3bfc81bba61a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:29.850163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08e463-9dde-46b5-a2ba-8bfd1b893008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:29.850784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08e463-9dde-46b5-a2ba-8bfd1b893008 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:29.851065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08e463-9dde-46b5-a2ba-8bfd1b893008 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:29.851338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08e463-9dde-46b5-a2ba-8bfd1b893008 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:29.852072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08e463-9dde-46b5-a2ba-8bfd1b893008 None admin] RBAC: Adding query filter params (role.id=c70d64f54bff4d2ba00749c556be9c49, scope.project.id=3d52ac8525814206ade19cf3bc6b9271, user.id=8e67b785954f4fd4b095552017407c73) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:29.852385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08e463-9dde-46b5-a2ba-8bfd1b893008 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:29.854031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08e463-9dde-46b5-a2ba-8bfd1b893008 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:29.870436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 80/161] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:29 2022] GET /identity/v3/role_assignments?user.id=8e67b785954f4fd4b095552017407c73&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 608 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:29.892429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece53985-4564-4ced-aaa2-31eb7a2a8528 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:29.922333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d13e03-616f-469c-b0e4-828e66f130a5 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:29.922634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d13e03-616f-469c-b0e4-828e66f130a5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:29.922889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d13e03-616f-469c-b0e4-828e66f130a5 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:29.924364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 81/162] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:29:29.938595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf80e6-983b-4fb9-8ca4-ea7af9ce5d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:29.939280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf80e6-983b-4fb9-8ca4-ea7af9ce5d3d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:29.939578 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf80e6-983b-4fb9-8ca4-ea7af9ce5d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:29.939896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf80e6-983b-4fb9-8ca4-ea7af9ce5d3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:29.944299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac14c7f-6b68-47cd-9250-1e5058aad392 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:29.944666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac14c7f-6b68-47cd-9250-1e5058aad392 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:29.944891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac14c7f-6b68-47cd-9250-1e5058aad392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:29.985710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fac14c7f-6b68-47cd-9250-1e5058aad392 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:29.986884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fac14c7f-6b68-47cd-9250-1e5058aad392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:30.106086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf80e6-983b-4fb9-8ca4-ea7af9ce5d3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:30.107596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf80e6-983b-4fb9-8ca4-ea7af9ce5d3d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:30.112069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 82/163] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:30.173826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:30.274949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 82/164] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 11:29:30.281087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac14c7f-6b68-47cd-9250-1e5058aad392 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:30.297903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35435b5-7260-4dbe-af49-c750c1b23a7f None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:30.298219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35435b5-7260-4dbe-af49-c750c1b23a7f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:30.298519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35435b5-7260-4dbe-af49-c750c1b23a7f None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:30.299851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 83/165] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:29:30.304935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd049168-4eea-47f2-850a-b5fdfd4f1579 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:30.305230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd049168-4eea-47f2-850a-b5fdfd4f1579 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:30.305509 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd049168-4eea-47f2-850a-b5fdfd4f1579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:30.329499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cd049168-4eea-47f2-850a-b5fdfd4f1579 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:30.330703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cd049168-4eea-47f2-850a-b5fdfd4f1579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:30.422505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:30.448612 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac14c7f-6b68-47cd-9250-1e5058aad392 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:30.471111 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 84/166] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:30.502582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2dfdb-e937-440a-9197-07fb6103849d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:30.503220 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2dfdb-e937-440a-9197-07fb6103849d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:30.503502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2dfdb-e937-440a-9197-07fb6103849d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:30.503777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2dfdb-e937-440a-9197-07fb6103849d None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:30.504927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2dfdb-e937-440a-9197-07fb6103849d None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:30.506370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2dfdb-e937-440a-9197-07fb6103849d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:30.508007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 83/167] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:30 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:30.514697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd049168-4eea-47f2-850a-b5fdfd4f1579 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:30.595812 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:30.640224 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd049168-4eea-47f2-850a-b5fdfd4f1579 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:30.682661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed495341-ed4b-4144-97de-f3008917d4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:30.683841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed495341-ed4b-4144-97de-f3008917d4aa None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:30.684117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed495341-ed4b-4144-97de-f3008917d4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:30.684388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed495341-ed4b-4144-97de-f3008917d4aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:30.685165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed495341-ed4b-4144-97de-f3008917d4aa None admin] RBAC: Adding query filter params (role.id=6f9c4ad710e84eaabc415a1dd1541055, scope.project.id=3d52ac8525814206ade19cf3bc6b9271, user.id=8e67b785954f4fd4b095552017407c73) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:30.685524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed495341-ed4b-4144-97de-f3008917d4aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:30.687052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed495341-ed4b-4144-97de-f3008917d4aa None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:30.693790 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 85/168] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:30 2022] GET /identity/v3/role_assignments?user.id=8e67b785954f4fd4b095552017407c73&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=6f9c4ad710e84eaabc415a1dd1541055 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:30.701669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2dfdb-e937-440a-9197-07fb6103849d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:30.855744 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2dfdb-e937-440a-9197-07fb6103849d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:30.868338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6c266-8213-432b-aeac-41208c000742 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:30.869043 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6c266-8213-432b-aeac-41208c000742 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:30.869536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6c266-8213-432b-aeac-41208c000742 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:30.871749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 86/169] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:30.877316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f15ed-135a-459f-b7e3-1242bf213bc4 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:30.877862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f15ed-135a-459f-b7e3-1242bf213bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:30.878351 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f15ed-135a-459f-b7e3-1242bf213bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:30.900636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d84f15ed-135a-459f-b7e3-1242bf213bc4 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:30.904378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d84f15ed-135a-459f-b7e3-1242bf213bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:30.922970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb4eb1-5dc9-4242-9721-cac9a117e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:30.923965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb4eb1-5dc9-4242-9721-cac9a117e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:30.924507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb4eb1-5dc9-4242-9721-cac9a117e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:30.924975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb4eb1-5dc9-4242-9721-cac9a117e6d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:30.946884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb4eb1-5dc9-4242-9721-cac9a117e6d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-99fb4eb1-5dc9-4242-9721-cac9a117e6d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:30.955639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:30.956708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:30.957523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:30.962685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 84/170] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:29:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:30.966983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb4eb1-5dc9-4242-9721-cac9a117e6d3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:30.987452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:31.047378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 87/171] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:31.107372 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb4eb1-5dc9-4242-9721-cac9a117e6d3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:31.135993 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f15ed-135a-459f-b7e3-1242bf213bc4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:31.144481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0479e0a-97af-4e3c-9f1d-ed9ba55dc4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:31.145230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0479e0a-97af-4e3c-9f1d-ed9ba55dc4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:31.145507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0479e0a-97af-4e3c-9f1d-ed9ba55dc4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:31.145717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0479e0a-97af-4e3c-9f1d-ed9ba55dc4f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:31.146387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0479e0a-97af-4e3c-9f1d-ed9ba55dc4f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:31.146687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0479e0a-97af-4e3c-9f1d-ed9ba55dc4f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:31.148063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0479e0a-97af-4e3c-9f1d-ed9ba55dc4f8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:31.157950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 85/172] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:31.164745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0479e0a-97af-4e3c-9f1d-ed9ba55dc4f8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:31.246946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:31.282675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0479e0a-97af-4e3c-9f1d-ed9ba55dc4f8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:31.313901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:31.335179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ada7-6481-496b-a170-4ca65232d046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:31.336233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ada7-6481-496b-a170-4ca65232d046 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:31.336700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ada7-6481-496b-a170-4ca65232d046 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:31.337120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ada7-6481-496b-a170-4ca65232d046 None admin] PATH_INFO: `/v3/users/824ec611f32e40609c56e94f7edd552e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:31.341164 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533ada7-6481-496b-a170-4ca65232d046 None admin] RBAC: Authorizing `identity:get_user(user_id=824ec611f32e40609c56e94f7edd552e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:31.342797 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533ada7-6481-496b-a170-4ca65232d046 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:31.344793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 86/173] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:31 2022] GET /identity/v3/users/824ec611f32e40609c56e94f7edd552e => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:31.350688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ada7-6481-496b-a170-4ca65232d046 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:31.451367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f15ed-135a-459f-b7e3-1242bf213bc4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:31.487300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ada7-6481-496b-a170-4ca65232d046 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:31.488157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5e989-9b75-4fbb-856f-46613d31de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:31.488821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e989-9b75-4fbb-856f-46613d31de42 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:31.489105 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e989-9b75-4fbb-856f-46613d31de42 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:31.489386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e989-9b75-4fbb-856f-46613d31de42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:31.500857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5e989-9b75-4fbb-856f-46613d31de42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-a6b5e989-9b75-4fbb-856f-46613d31de42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:31.506033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:31.506912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:31.507662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 88/174] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:31.508310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5e989-9b75-4fbb-856f-46613d31de42 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:31.534028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916cf98-73d6-4f7f-92d7-84dc92bdeb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:31.535401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916cf98-73d6-4f7f-92d7-84dc92bdeb2d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:31.536104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916cf98-73d6-4f7f-92d7-84dc92bdeb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:31.536633 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916cf98-73d6-4f7f-92d7-84dc92bdeb2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:31.572877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:31.619890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916cf98-73d6-4f7f-92d7-84dc92bdeb2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:31.623123 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916cf98-73d6-4f7f-92d7-84dc92bdeb2d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:31.659145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5e989-9b75-4fbb-856f-46613d31de42 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:31.688073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:31.703131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687e509-6b1b-42d8-82cb-06d646103ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:31.703908 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687e509-6b1b-42d8-82cb-06d646103ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:31.704273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687e509-6b1b-42d8-82cb-06d646103ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:31.704727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687e509-6b1b-42d8-82cb-06d646103ad8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:31.716283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687e509-6b1b-42d8-82cb-06d646103ad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-e687e509-6b1b-42d8-82cb-06d646103ad8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:31.721442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:31.722354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:31.723148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 89/175] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:29:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:31.723783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687e509-6b1b-42d8-82cb-06d646103ad8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:31.758981 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 87/176] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:31.761043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2530-f892-4036-8dcd-8cda75fdd18a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:31.761870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2530-f892-4036-8dcd-8cda75fdd18a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:31.762178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2530-f892-4036-8dcd-8cda75fdd18a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:31.763712 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 88/177] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:31.764308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a2530-f892-4036-8dcd-8cda75fdd18a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:31.812193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:31.852616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687e509-6b1b-42d8-82cb-06d646103ad8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:31.876794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a2530-f892-4036-8dcd-8cda75fdd18a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:31.893013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d1ce7-d342-4ed8-a305-0debe0b311a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:31.893786 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d1ce7-d342-4ed8-a305-0debe0b311a0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:31.894069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d1ce7-d342-4ed8-a305-0debe0b311a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:31.894639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d1ce7-d342-4ed8-a305-0debe0b311a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:31.895317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d1ce7-d342-4ed8-a305-0debe0b311a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:31.895621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d1ce7-d342-4ed8-a305-0debe0b311a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:31.897517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d1ce7-d342-4ed8-a305-0debe0b311a0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:31.906867 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 90/178] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:31.912032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbe902-5c75-469d-bdfc-e16d4a3550eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:31.912563 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496cf6b-10fa-4d7c-9295-c8ac0fe69f34 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:31.912665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbe902-5c75-469d-bdfc-e16d4a3550eb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:31.912859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496cf6b-10fa-4d7c-9295-c8ac0fe69f34 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:31.912970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbe902-5c75-469d-bdfc-e16d4a3550eb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:31.913160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496cf6b-10fa-4d7c-9295-c8ac0fe69f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:31.913260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbe902-5c75-469d-bdfc-e16d4a3550eb None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:31.913827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbe902-5c75-469d-bdfc-e16d4a3550eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:31.914127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbe902-5c75-469d-bdfc-e16d4a3550eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:31.915490 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbe902-5c75-469d-bdfc-e16d4a3550eb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:31.928521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 89/179] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:31.928521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbe902-5c75-469d-bdfc-e16d4a3550eb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:31.935837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1496cf6b-10fa-4d7c-9295-c8ac0fe69f34 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:31.937087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1496cf6b-10fa-4d7c-9295-c8ac0fe69f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:31.974646 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:32.007588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 91/180] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:32.014298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e9606-5396-4854-9787-a5d775427ad8 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.014588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e9606-5396-4854-9787-a5d775427ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.014870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e9606-5396-4854-9787-a5d775427ad8 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.016451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 92/181] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:32.017251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e9606-5396-4854-9787-a5d775427ad8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:32.038254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbe902-5c75-469d-bdfc-e16d4a3550eb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:32.040297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:32.079851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688864dc-0ae6-493e-b4e7-dba84e630034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:32.080887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688864dc-0ae6-493e-b4e7-dba84e630034 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.081388 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688864dc-0ae6-493e-b4e7-dba84e630034 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.081852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688864dc-0ae6-493e-b4e7-dba84e630034 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.083160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688864dc-0ae6-493e-b4e7-dba84e630034 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:32.084832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688864dc-0ae6-493e-b4e7-dba84e630034 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:32.086678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 90/182] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:31 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:32.087709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688864dc-0ae6-493e-b4e7-dba84e630034 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:32.164417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e9606-5396-4854-9787-a5d775427ad8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:32.199811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47469120-94fe-4297-a153-73654ca5da82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:32.200012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688864dc-0ae6-493e-b4e7-dba84e630034 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:32.200441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47469120-94fe-4297-a153-73654ca5da82 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.200751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47469120-94fe-4297-a153-73654ca5da82 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.201026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47469120-94fe-4297-a153-73654ca5da82 None admin] PATH_INFO: `/v3/users/8e67b785954f4fd4b095552017407c73` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.202163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47469120-94fe-4297-a153-73654ca5da82 None admin] RBAC: Authorizing `identity:get_user(user_id=8e67b785954f4fd4b095552017407c73)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:32.203635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47469120-94fe-4297-a153-73654ca5da82 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:32.205327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 93/183] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:32 2022] GET /identity/v3/users/8e67b785954f4fd4b095552017407c73 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:32.206441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31ff7e-8aaf-4930-9ec1-27b678b7bae1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.206721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31ff7e-8aaf-4930-9ec1-27b678b7bae1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.206994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31ff7e-8aaf-4930-9ec1-27b678b7bae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.225035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4b31ff7e-8aaf-4930-9ec1-27b678b7bae1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:32.226172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4b31ff7e-8aaf-4930-9ec1-27b678b7bae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:32.239888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fabcb-3a7e-489a-8ae9-1944571201b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:32.240483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fabcb-3a7e-489a-8ae9-1944571201b3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.240791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fabcb-3a7e-489a-8ae9-1944571201b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.241259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fabcb-3a7e-489a-8ae9-1944571201b3 None admin] PATH_INFO: `/v3/users/b227303aa9ee413c815f7c3ab94a0845` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.255702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483fabcb-3a7e-489a-8ae9-1944571201b3 None admin] RBAC: Authorizing `identity:get_user(user_id=b227303aa9ee413c815f7c3ab94a0845)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:32.257197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483fabcb-3a7e-489a-8ae9-1944571201b3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:32.258837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 91/184] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:32 2022] GET /identity/v3/users/b227303aa9ee413c815f7c3ab94a0845 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:32.263222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:32.264947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fabcb-3a7e-489a-8ae9-1944571201b3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:32.293468 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:32.299479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 94/185] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:32.300753 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f42a5-b940-480e-b0ee-9b4754888f63 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.301041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f42a5-b940-480e-b0ee-9b4754888f63 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.301316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f42a5-b940-480e-b0ee-9b4754888f63 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.302746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 95/186] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:32.307244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f42a5-b940-480e-b0ee-9b4754888f63 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:32.371859 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fabcb-3a7e-489a-8ae9-1944571201b3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:32.408772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba14ad-26da-4fff-87ff-d75c936ed7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:32.409377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba14ad-26da-4fff-87ff-d75c936ed7ff None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.409662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba14ad-26da-4fff-87ff-d75c936ed7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.409935 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba14ad-26da-4fff-87ff-d75c936ed7ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.410687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba14ad-26da-4fff-87ff-d75c936ed7ff None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=3d52ac8525814206ade19cf3bc6b9271, user.id=824ec611f32e40609c56e94f7edd552e) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:32.410995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba14ad-26da-4fff-87ff-d75c936ed7ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:32.412430 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba14ad-26da-4fff-87ff-d75c936ed7ff None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:32.418614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 92/187] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:32 2022] GET /identity/v3/role_assignments?user.id=824ec611f32e40609c56e94f7edd552e&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:32.419414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba14ad-26da-4fff-87ff-d75c936ed7ff None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:32.432747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f42a5-b940-480e-b0ee-9b4754888f63 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:32.482692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c2ba2-3612-4f84-8cf7-bc9543e8eb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:32.483639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c2ba2-3612-4f84-8cf7-bc9543e8eb04 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.484010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c2ba2-3612-4f84-8cf7-bc9543e8eb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.484359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c2ba2-3612-4f84-8cf7-bc9543e8eb04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.543137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:32.555526 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:32.569749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba14ad-26da-4fff-87ff-d75c936ed7ff None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:32.588753 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c2ba2-3612-4f84-8cf7-bc9543e8eb04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:32.590327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c2ba2-3612-4f84-8cf7-bc9543e8eb04 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:32.615901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4faad-04dd-4dcf-8f4c-d91515e0a3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:32.616637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4faad-04dd-4dcf-8f4c-d91515e0a3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.616920 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4faad-04dd-4dcf-8f4c-d91515e0a3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.617193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4faad-04dd-4dcf-8f4c-d91515e0a3c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.691213 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4faad-04dd-4dcf-8f4c-d91515e0a3c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:32.692699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4faad-04dd-4dcf-8f4c-d91515e0a3c7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:32.697199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 96/188] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:32.697199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c2ba2-3612-4f84-8cf7-bc9543e8eb04 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:32.768056 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:32.781562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:32.790844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 93/189] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:32.790844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c858a-b303-4a8c-b139-84ef7fc3b430 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.790844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c858a-b303-4a8c-b139-84ef7fc3b430 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.790844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c858a-b303-4a8c-b139-84ef7fc3b430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.804144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c2ba2-3612-4f84-8cf7-bc9543e8eb04 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:32.810298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-941c858a-b303-4a8c-b139-84ef7fc3b430 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:32.811462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-941c858a-b303-4a8c-b139-84ef7fc3b430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:32.837812 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f9c09-663d-4858-9674-32d6d1ad5635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:32.838439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f9c09-663d-4858-9674-32d6d1ad5635 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.838716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f9c09-663d-4858-9674-32d6d1ad5635 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.838989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f9c09-663d-4858-9674-32d6d1ad5635 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.849542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f9c09-663d-4858-9674-32d6d1ad5635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-711f9c09-663d-4858-9674-32d6d1ad5635 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:32.854383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:32.855562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 97/190] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:29:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:32.856435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f9c09-663d-4858-9674-32d6d1ad5635 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:32.883375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 94/191] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:32.892764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c858a-b303-4a8c-b139-84ef7fc3b430 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:32.968967 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f9c09-663d-4858-9674-32d6d1ad5635 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:32.990968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c858a-b303-4a8c-b139-84ef7fc3b430 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:32.998100 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:33.005174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8c50c-2957-4e98-bb3d-b971df9a86e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.005799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8c50c-2957-4e98-bb3d-b971df9a86e6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.006077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8c50c-2957-4e98-bb3d-b971df9a86e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.006353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8c50c-2957-4e98-bb3d-b971df9a86e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.016118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:33.018913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8c50c-2957-4e98-bb3d-b971df9a86e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-e1a8c50c-2957-4e98-bb3d-b971df9a86e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:33.023860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:33.024770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:33.025555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 98/192] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:33.026209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8c50c-2957-4e98-bb3d-b971df9a86e6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:33.033766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a639a3-3f0d-4bf1-a7a6-593a0161bbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.034398 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a639a3-3f0d-4bf1-a7a6-593a0161bbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.034734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a639a3-3f0d-4bf1-a7a6-593a0161bbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.035006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a639a3-3f0d-4bf1-a7a6-593a0161bbb3 None admin] PATH_INFO: `/v3/projects/b5a5c662c30541dd9d9b9e1140540e19` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.036113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a639a3-3f0d-4bf1-a7a6-593a0161bbb3 None admin] RBAC: Authorizing `identity:get_project(project_id=b5a5c662c30541dd9d9b9e1140540e19)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:33.037591 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a639a3-3f0d-4bf1-a7a6-593a0161bbb3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:33.039329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 95/193] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:32 2022] GET /identity/v3/projects/b5a5c662c30541dd9d9b9e1140540e19 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:33.040012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a639a3-3f0d-4bf1-a7a6-593a0161bbb3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:33.139072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8c50c-2957-4e98-bb3d-b971df9a86e6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:33.153146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a639a3-3f0d-4bf1-a7a6-593a0161bbb3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:33.176371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13b24c-77f5-482e-aa6e-d24c3227691d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.177029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13b24c-77f5-482e-aa6e-d24c3227691d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.177309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13b24c-77f5-482e-aa6e-d24c3227691d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.177580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13b24c-77f5-482e-aa6e-d24c3227691d None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.178705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13b24c-77f5-482e-aa6e-d24c3227691d None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:33.180153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13b24c-77f5-482e-aa6e-d24c3227691d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:33.181807 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 99/194] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:33 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:33.185385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13b24c-77f5-482e-aa6e-d24c3227691d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:33.189958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabd95f-4009-49cf-9946-ccfca463807c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.190656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabd95f-4009-49cf-9946-ccfca463807c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.191193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabd95f-4009-49cf-9946-ccfca463807c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.191465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabd95f-4009-49cf-9946-ccfca463807c None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.192157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabd95f-4009-49cf-9946-ccfca463807c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:33.192462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabd95f-4009-49cf-9946-ccfca463807c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:33.193848 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabd95f-4009-49cf-9946-ccfca463807c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:33.206848 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 96/195] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:29:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:33.206848 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabd95f-4009-49cf-9946-ccfca463807c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:33.248951 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:33.255753 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:33.289197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13b24c-77f5-482e-aa6e-d24c3227691d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:33.303182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabd95f-4009-49cf-9946-ccfca463807c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:33.323766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019d4ee-d77e-4c2d-8c52-e5e52046c8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.324468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019d4ee-d77e-4c2d-8c52-e5e52046c8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.324809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019d4ee-d77e-4c2d-8c52-e5e52046c8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.325084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019d4ee-d77e-4c2d-8c52-e5e52046c8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.325759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019d4ee-d77e-4c2d-8c52-e5e52046c8a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:33.326064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019d4ee-d77e-4c2d-8c52-e5e52046c8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:33.327494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019d4ee-d77e-4c2d-8c52-e5e52046c8a2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:33.335606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 100/196] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:33.336174 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9155409-f947-4b88-bcfa-a0112be4e850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019d4ee-d77e-4c2d-8c52-e5e52046c8a2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:33.336800 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9155409-f947-4b88-bcfa-a0112be4e850 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.337078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9155409-f947-4b88-bcfa-a0112be4e850 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.337349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9155409-f947-4b88-bcfa-a0112be4e850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.338064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9155409-f947-4b88-bcfa-a0112be4e850 None admin] RBAC: Adding query filter params (role.id=c70d64f54bff4d2ba00749c556be9c49, scope.project.id=b5a5c662c30541dd9d9b9e1140540e19, user.id=8e67b785954f4fd4b095552017407c73) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:33.338371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9155409-f947-4b88-bcfa-a0112be4e850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:33.339848 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9155409-f947-4b88-bcfa-a0112be4e850 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:33.345737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 97/197] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:33 2022] GET /identity/v3/role_assignments?user.id=8e67b785954f4fd4b095552017407c73&scope.project.id=b5a5c662c30541dd9d9b9e1140540e19&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:33.346371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9155409-f947-4b88-bcfa-a0112be4e850 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:33.459831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019d4ee-d77e-4c2d-8c52-e5e52046c8a2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:33.461473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9155409-f947-4b88-bcfa-a0112be4e850 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:33.480109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:33.484305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:33.503630 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef7d41-dda4-4157-840c-8d22bf2c05a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.503851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d19525-4abc-4c8f-af66-d1b1785e7d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.504510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d19525-4abc-4c8f-af66-d1b1785e7d92 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.504639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef7d41-dda4-4157-840c-8d22bf2c05a7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.504878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d19525-4abc-4c8f-af66-d1b1785e7d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.505079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef7d41-dda4-4157-840c-8d22bf2c05a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.505165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d19525-4abc-4c8f-af66-d1b1785e7d92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.505518 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef7d41-dda4-4157-840c-8d22bf2c05a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.506710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef7d41-dda4-4157-840c-8d22bf2c05a7 None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c, user.id=b227303aa9ee413c815f7c3ab94a0845) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:33.506821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d19525-4abc-4c8f-af66-d1b1785e7d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:33.506918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef7d41-dda4-4157-840c-8d22bf2c05a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:33.508409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d19525-4abc-4c8f-af66-d1b1785e7d92 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:33.508551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef7d41-dda4-4157-840c-8d22bf2c05a7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:33.509857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 101/198] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:33.515674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d19525-4abc-4c8f-af66-d1b1785e7d92 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:33.522694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 98/199] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:33 2022] GET /identity/v3/role_assignments?user.id=b227303aa9ee413c815f7c3ab94a0845&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:33.522694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef7d41-dda4-4157-840c-8d22bf2c05a7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:33.646256 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef7d41-dda4-4157-840c-8d22bf2c05a7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:33.649557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d19525-4abc-4c8f-af66-d1b1785e7d92 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:33.687704 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90133f4a-b51a-4590-aa66-33187a6b82e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.688354 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90133f4a-b51a-4590-aa66-33187a6b82e6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.688650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90133f4a-b51a-4590-aa66-33187a6b82e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.688930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90133f4a-b51a-4590-aa66-33187a6b82e6 None admin] PATH_INFO: `/v3/users/29822e5b266140fab42679a3fe2533fb` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.690068 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90133f4a-b51a-4590-aa66-33187a6b82e6 None admin] RBAC: Authorizing `identity:get_user(user_id=29822e5b266140fab42679a3fe2533fb)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:33.691774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90133f4a-b51a-4590-aa66-33187a6b82e6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:33.698399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 99/200] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:33 2022] GET /identity/v3/users/29822e5b266140fab42679a3fe2533fb => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:33.698399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90133f4a-b51a-4590-aa66-33187a6b82e6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:33.703471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba80732-b5cf-4498-967e-c0190b258c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.704119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba80732-b5cf-4498-967e-c0190b258c1d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.704396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba80732-b5cf-4498-967e-c0190b258c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.704698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba80732-b5cf-4498-967e-c0190b258c1d None admin] PATH_INFO: `/v3/users/b227303aa9ee413c815f7c3ab94a0845` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.706029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba80732-b5cf-4498-967e-c0190b258c1d None admin] RBAC: Authorizing `identity:get_user(user_id=b227303aa9ee413c815f7c3ab94a0845)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:33.707518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba80732-b5cf-4498-967e-c0190b258c1d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:33.709162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 102/201] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:33 2022] GET /identity/v3/users/b227303aa9ee413c815f7c3ab94a0845 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:33.714027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba80732-b5cf-4498-967e-c0190b258c1d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:33.763041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:33.771660 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:33.819346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90133f4a-b51a-4590-aa66-33187a6b82e6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:33.820596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba80732-b5cf-4498-967e-c0190b258c1d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:33.856259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151beb91-12e8-4970-be6a-b7fec1f0aa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.856910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151beb91-12e8-4970-be6a-b7fec1f0aa93 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.857187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151beb91-12e8-4970-be6a-b7fec1f0aa93 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.857460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151beb91-12e8-4970-be6a-b7fec1f0aa93 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.858108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151beb91-12e8-4970-be6a-b7fec1f0aa93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:33.858291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8ebd3-4021-416d-922d-79e19066ca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:33.858883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ebd3-4021-416d-922d-79e19066ca76 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:33.859162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ebd3-4021-416d-922d-79e19066ca76 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:33.859436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ebd3-4021-416d-922d-79e19066ca76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:33.859585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151beb91-12e8-4970-be6a-b7fec1f0aa93 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:33.880942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 100/202] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:29:33 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:33.881844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151beb91-12e8-4970-be6a-b7fec1f0aa93 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:33.955765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8ebd3-4021-416d-922d-79e19066ca76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:33.957734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8ebd3-4021-416d-922d-79e19066ca76 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:34.001243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151beb91-12e8-4970-be6a-b7fec1f0aa93 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:34.003915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:34.031401 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:34.050931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f6310-75bb-4842-b7e4-b7c65f04be86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:34.051542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f6310-75bb-4842-b7e4-b7c65f04be86 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:34.051822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f6310-75bb-4842-b7e4-b7c65f04be86 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:34.052095 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f6310-75bb-4842-b7e4-b7c65f04be86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:34.098225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 103/203] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:34.104267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8ebd3-4021-416d-922d-79e19066ca76 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:34.142108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f6310-75bb-4842-b7e4-b7c65f04be86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:34.143673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f6310-75bb-4842-b7e4-b7c65f04be86 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:34.234186 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8ebd3-4021-416d-922d-79e19066ca76 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:34.272122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 101/204] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:34.279035 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f6310-75bb-4842-b7e4-b7c65f04be86 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:34.282014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33961065-a469-4ae2-8dd1-b2615f701b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:34.282741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33961065-a469-4ae2-8dd1-b2615f701b4b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:34.283097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33961065-a469-4ae2-8dd1-b2615f701b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:34.283198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:34.283798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33961065-a469-4ae2-8dd1-b2615f701b4b None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:34.285050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33961065-a469-4ae2-8dd1-b2615f701b4b None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:34.286528 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33961065-a469-4ae2-8dd1-b2615f701b4b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:34.288683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 104/205] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:34 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:34.296267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33961065-a469-4ae2-8dd1-b2615f701b4b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:34.328344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:34.398867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f6310-75bb-4842-b7e4-b7c65f04be86 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:34.413616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33961065-a469-4ae2-8dd1-b2615f701b4b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:34.439700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6d99b-cee7-41f1-9e2b-50c14c58968e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:34.440337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6d99b-cee7-41f1-9e2b-50c14c58968e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:34.440656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6d99b-cee7-41f1-9e2b-50c14c58968e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:34.441198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6d99b-cee7-41f1-9e2b-50c14c58968e None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:34.442339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6d99b-cee7-41f1-9e2b-50c14c58968e None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:34.443794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6d99b-cee7-41f1-9e2b-50c14c58968e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:34.445683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 102/206] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:34 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:34.450874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91121f42-9990-4189-9ccc-685e475ce0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:34.451490 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91121f42-9990-4189-9ccc-685e475ce0aa None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:34.451680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6d99b-cee7-41f1-9e2b-50c14c58968e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:34.451789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91121f42-9990-4189-9ccc-685e475ce0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:34.452024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91121f42-9990-4189-9ccc-685e475ce0aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:34.452783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91121f42-9990-4189-9ccc-685e475ce0aa None admin] RBAC: Adding query filter params (role.id=4febe0fde4a74359af9325b7352b75d6, scope.project.id=3d52ac8525814206ade19cf3bc6b9271, user.id=29822e5b266140fab42679a3fe2533fb) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:34.453092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91121f42-9990-4189-9ccc-685e475ce0aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:34.454528 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91121f42-9990-4189-9ccc-685e475ce0aa None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:34.460672 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 105/207] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:34 2022] GET /identity/v3/role_assignments?user.id=29822e5b266140fab42679a3fe2533fb&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=4febe0fde4a74359af9325b7352b75d6 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:34.535511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:34.569659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6d99b-cee7-41f1-9e2b-50c14c58968e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:34.611916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256abc5-e659-47d9-bf59-2388486c75ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:34.612573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256abc5-e659-47d9-bf59-2388486c75ad None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:34.612864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256abc5-e659-47d9-bf59-2388486c75ad None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:34.613146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256abc5-e659-47d9-bf59-2388486c75ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:34.613898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256abc5-e659-47d9-bf59-2388486c75ad None admin] RBAC: Adding query filter params (role.id=6f9c4ad710e84eaabc415a1dd1541055, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c, user.id=b227303aa9ee413c815f7c3ab94a0845) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:34.614210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256abc5-e659-47d9-bf59-2388486c75ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:34.615706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256abc5-e659-47d9-bf59-2388486c75ad None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:34.623239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 103/208] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:34 2022] GET /identity/v3/role_assignments?user.id=b227303aa9ee413c815f7c3ab94a0845&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=6f9c4ad710e84eaabc415a1dd1541055 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:35.657419 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eae1f-faae-4364-963f-cb90e4f17158 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:35.658434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eae1f-faae-4364-963f-cb90e4f17158 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:35.658932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eae1f-faae-4364-963f-cb90e4f17158 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:35.660885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 106/209] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:35.666499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb99e9-82da-480b-b6a1-f6efa4b06769 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:35.667348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb99e9-82da-480b-b6a1-f6efa4b06769 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:35.668339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb99e9-82da-480b-b6a1-f6efa4b06769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:35.696019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a9fb99e9-82da-480b-b6a1-f6efa4b06769 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:35.697550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a9fb99e9-82da-480b-b6a1-f6efa4b06769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:35.772622 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 104/210] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:36.037675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83eae1f-faae-4364-963f-cb90e4f17158 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:36.136957 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:36.151061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc7342-b1a1-4e8f-abec-7d9c26cf1a50 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.151678 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83eae1f-faae-4364-963f-cb90e4f17158 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:36.152619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc7342-b1a1-4e8f-abec-7d9c26cf1a50 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.153087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc7342-b1a1-4e8f-abec-7d9c26cf1a50 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.154754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 105/211] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:36.160260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd607b-6ba0-4fc6-bcee-60905cc5f210 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.160813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd607b-6ba0-4fc6-bcee-60905cc5f210 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.161282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd607b-6ba0-4fc6-bcee-60905cc5f210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.181517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7fcd607b-6ba0-4fc6-bcee-60905cc5f210 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:36.184045 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7fcd607b-6ba0-4fc6-bcee-60905cc5f210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:36.193342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c4a05-7d7e-424b-addd-4546d3020fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:36.194000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4a05-7d7e-424b-addd-4546d3020fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.194286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4a05-7d7e-424b-addd-4546d3020fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.194566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4a05-7d7e-424b-addd-4546d3020fc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.209636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c4a05-7d7e-424b-addd-4546d3020fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-ee4c4a05-7d7e-424b-addd-4546d3020fc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:36.219601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:36.220552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:36.226037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 107/212] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:29:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:36.231642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c4a05-7d7e-424b-addd-4546d3020fc3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:36.279948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:36.299179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 106/213] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:36.299179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d544df5-9b29-473e-ac0f-99c669d0e3fe None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.299179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d544df5-9b29-473e-ac0f-99c669d0e3fe None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.299179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d544df5-9b29-473e-ac0f-99c669d0e3fe None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.299179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 107/214] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:36.304651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638bda3-353f-4075-8085-201d7bde7282 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.304940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638bda3-353f-4075-8085-201d7bde7282 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.307007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638bda3-353f-4075-8085-201d7bde7282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.344677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f638bda3-353f-4075-8085-201d7bde7282 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:36.346603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f638bda3-353f-4075-8085-201d7bde7282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:36.402614 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c4a05-7d7e-424b-addd-4546d3020fc3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:36.446226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a7897-650d-4cbd-9f8f-275b8c082096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:36.446852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a7897-650d-4cbd-9f8f-275b8c082096 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.447387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a7897-650d-4cbd-9f8f-275b8c082096 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.447670 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a7897-650d-4cbd-9f8f-275b8c082096 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.448357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a7897-650d-4cbd-9f8f-275b8c082096 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:36.449292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a7897-650d-4cbd-9f8f-275b8c082096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:36.451425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a7897-650d-4cbd-9f8f-275b8c082096 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:36.461492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 108/215] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:36.462707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 108/216] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:36.469553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a7897-650d-4cbd-9f8f-275b8c082096 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:36.516856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:36.590502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638bda3-353f-4075-8085-201d7bde7282 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:36.667313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a7897-650d-4cbd-9f8f-275b8c082096 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:36.712159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df220c24-410d-45f9-90af-676b426ff68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:36.713450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220c24-410d-45f9-90af-676b426ff68d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.713738 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220c24-410d-45f9-90af-676b426ff68d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.715320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220c24-410d-45f9-90af-676b426ff68d None admin] PATH_INFO: `/v3/users/b227303aa9ee413c815f7c3ab94a0845` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.716531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df220c24-410d-45f9-90af-676b426ff68d None admin] RBAC: Authorizing `identity:get_user(user_id=b227303aa9ee413c815f7c3ab94a0845)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:36.718000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df220c24-410d-45f9-90af-676b426ff68d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:36.720682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 109/217] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:36 2022] GET /identity/v3/users/b227303aa9ee413c815f7c3ab94a0845 => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:36.721188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa47c8-2551-4208-b8bf-d610ab6fe580 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.721475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa47c8-2551-4208-b8bf-d610ab6fe580 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.721749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa47c8-2551-4208-b8bf-d610ab6fe580 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.723751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 110/218] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:36.726466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa47c8-2551-4208-b8bf-d610ab6fe580 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:36.751577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638bda3-353f-4075-8085-201d7bde7282 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:36.778164 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:36.786967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442435b-cd5b-4895-964b-57eb9e2cc8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:36.787596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442435b-cd5b-4895-964b-57eb9e2cc8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.787877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442435b-cd5b-4895-964b-57eb9e2cc8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.788152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442435b-cd5b-4895-964b-57eb9e2cc8d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.798596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442435b-cd5b-4895-964b-57eb9e2cc8d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a442435b-cd5b-4895-964b-57eb9e2cc8d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:36.803352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:36.804411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:36.805282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 109/219] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:29:36.809849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0798060-c29f-490e-8ccb-1bc0169b6dfb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.810129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0798060-c29f-490e-8ccb-1bc0169b6dfb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.810471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0798060-c29f-490e-8ccb-1bc0169b6dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.811552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:36.828856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d0798060-c29f-490e-8ccb-1bc0169b6dfb None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:36.830024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d0798060-c29f-490e-8ccb-1bc0169b6dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:36.835442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa47c8-2551-4208-b8bf-d610ab6fe580 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:36.868315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a26ce9-d7b1-47fd-866e-59c4a8c8ffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:36.868895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a26ce9-d7b1-47fd-866e-59c4a8c8ffc2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.869172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a26ce9-d7b1-47fd-866e-59c4a8c8ffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.869441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a26ce9-d7b1-47fd-866e-59c4a8c8ffc2 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.871088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a26ce9-d7b1-47fd-866e-59c4a8c8ffc2 None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:36.872871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a26ce9-d7b1-47fd-866e-59c4a8c8ffc2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:36.874652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 111/220] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:36 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:36.875245 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a26ce9-d7b1-47fd-866e-59c4a8c8ffc2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:36.899456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 110/221] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:36.900370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0798060-c29f-490e-8ccb-1bc0169b6dfb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:36.982164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a26ce9-d7b1-47fd-866e-59c4a8c8ffc2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:37.007365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0798060-c29f-490e-8ccb-1bc0169b6dfb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:37.015575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ad16c-20a2-4552-a9c7-3bbe25aea71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:37.016199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ad16c-20a2-4552-a9c7-3bbe25aea71d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.016477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ad16c-20a2-4552-a9c7-3bbe25aea71d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.016774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ad16c-20a2-4552-a9c7-3bbe25aea71d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.017871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ad16c-20a2-4552-a9c7-3bbe25aea71d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:37.019306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ad16c-20a2-4552-a9c7-3bbe25aea71d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:37.021048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 112/222] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:37.021687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ad16c-20a2-4552-a9c7-3bbe25aea71d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:37.029826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:37.041392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:37.052870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c15b8-cb92-4930-ba1a-6eae1ca9ca94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:37.053700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c15b8-cb92-4930-ba1a-6eae1ca9ca94 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.054044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c15b8-cb92-4930-ba1a-6eae1ca9ca94 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.054380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c15b8-cb92-4930-ba1a-6eae1ca9ca94 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.055105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c15b8-cb92-4930-ba1a-6eae1ca9ca94 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:37.055474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c15b8-cb92-4930-ba1a-6eae1ca9ca94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:37.056937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c15b8-cb92-4930-ba1a-6eae1ca9ca94 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:37.070716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 111/223] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:37.070716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c15b8-cb92-4930-ba1a-6eae1ca9ca94 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:37.140139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ad16c-20a2-4552-a9c7-3bbe25aea71d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:37.175813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292b914-1aac-45e4-8b7a-9b8389dde677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:37.176467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b914-1aac-45e4-8b7a-9b8389dde677 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.176779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b914-1aac-45e4-8b7a-9b8389dde677 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.176895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c15b8-cb92-4930-ba1a-6eae1ca9ca94 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:37.177053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b914-1aac-45e4-8b7a-9b8389dde677 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/users/b227303aa9ee413c815f7c3ab94a0845/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.178977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292b914-1aac-45e4-8b7a-9b8389dde677 None admin] RBAC: Authorizing `identity:create_grant(project_id=272ff39535e04ccb8b5d3e8facf8135c, user_id=b227303aa9ee413c815f7c3ab94a0845, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:37.180406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292b914-1aac-45e4-8b7a-9b8389dde677 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:37.194131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 113/224] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:29:37 2022] PUT /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/users/b227303aa9ee413c815f7c3ab94a0845/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:37.194931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292b914-1aac-45e4-8b7a-9b8389dde677 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:37.221698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd255b21-03a3-432f-a0c2-8d580d60a5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:37.222479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd255b21-03a3-432f-a0c2-8d580d60a5b8 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.222767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd255b21-03a3-432f-a0c2-8d580d60a5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.223040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd255b21-03a3-432f-a0c2-8d580d60a5b8 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.223821 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd255b21-03a3-432f-a0c2-8d580d60a5b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:37.225382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd255b21-03a3-432f-a0c2-8d580d60a5b8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:37.251316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 112/225] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:29:37 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:37.261684 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd255b21-03a3-432f-a0c2-8d580d60a5b8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:37.288992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:37.314681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:37.322420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292b914-1aac-45e4-8b7a-9b8389dde677 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:37.368087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729349f0-2114-4efe-b6af-64a48c33a3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:37.368929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729349f0-2114-4efe-b6af-64a48c33a3fe None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.369219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729349f0-2114-4efe-b6af-64a48c33a3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.369499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729349f0-2114-4efe-b6af-64a48c33a3fe None admin] PATH_INFO: `/v3/users/8a78b727aba946f9b76b64b4943b8bc5` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.392680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729349f0-2114-4efe-b6af-64a48c33a3fe None admin] RBAC: Authorizing `identity:get_user(user_id=8a78b727aba946f9b76b64b4943b8bc5)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:37.392680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729349f0-2114-4efe-b6af-64a48c33a3fe None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:37.393611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 114/226] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:37 2022] GET /identity/v3/users/8a78b727aba946f9b76b64b4943b8bc5 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:37.399163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729349f0-2114-4efe-b6af-64a48c33a3fe None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:37.408751 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd255b21-03a3-432f-a0c2-8d580d60a5b8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:37.456939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715360b-a674-4c8b-b562-baca0f91f9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:37.457898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715360b-a674-4c8b-b562-baca0f91f9ea None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.458465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715360b-a674-4c8b-b562-baca0f91f9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.458743 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715360b-a674-4c8b-b562-baca0f91f9ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.474784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3715360b-a674-4c8b-b562-baca0f91f9ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3715360b-a674-4c8b-b562-baca0f91f9ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:37.480317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:37.481320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:37.482134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 113/227] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:29:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:37.488564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715360b-a674-4c8b-b562-baca0f91f9ea None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:37.536322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729349f0-2114-4efe-b6af-64a48c33a3fe None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:37.572331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:37.580846 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f183d4-6adf-42b5-b3fc-4e276e17661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:37.581469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f183d4-6adf-42b5-b3fc-4e276e17661d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.581758 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f183d4-6adf-42b5-b3fc-4e276e17661d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.582038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f183d4-6adf-42b5-b3fc-4e276e17661d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.613261 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:37.621304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715360b-a674-4c8b-b562-baca0f91f9ea None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:37.666310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580d8f7-548c-4e0b-938d-72f9218592ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:37.667172 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580d8f7-548c-4e0b-938d-72f9218592ba None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.668103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580d8f7-548c-4e0b-938d-72f9218592ba None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.668390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580d8f7-548c-4e0b-938d-72f9218592ba None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.669072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580d8f7-548c-4e0b-938d-72f9218592ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:37.669363 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580d8f7-548c-4e0b-938d-72f9218592ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:37.670763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580d8f7-548c-4e0b-938d-72f9218592ba None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:37.678281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f183d4-6adf-42b5-b3fc-4e276e17661d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:37.679786 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f183d4-6adf-42b5-b3fc-4e276e17661d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:37.682544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 114/228] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:29:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:37.689855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580d8f7-548c-4e0b-938d-72f9218592ba None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:37.817101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 115/229] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 11:29:37.817689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580d8f7-548c-4e0b-938d-72f9218592ba None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:37.826458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f183d4-6adf-42b5-b3fc-4e276e17661d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:37.863513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c66e2b-0fe2-4dbd-a9f5-995df57917e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:37.864520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c66e2b-0fe2-4dbd-a9f5-995df57917e4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.865220 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c66e2b-0fe2-4dbd-a9f5-995df57917e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.865654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c66e2b-0fe2-4dbd-a9f5-995df57917e4 None admin] PATH_INFO: `/v3/users/b227303aa9ee413c815f7c3ab94a0845` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.867682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c66e2b-0fe2-4dbd-a9f5-995df57917e4 None admin] RBAC: Authorizing `identity:get_user(user_id=b227303aa9ee413c815f7c3ab94a0845)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:37.869691 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:37.871311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c66e2b-0fe2-4dbd-a9f5-995df57917e4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:37.873869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 115/230] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:37 2022] GET /identity/v3/users/b227303aa9ee413c815f7c3ab94a0845 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:37.878905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c66e2b-0fe2-4dbd-a9f5-995df57917e4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:37.913540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:37.948336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f183d4-6adf-42b5-b3fc-4e276e17661d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:37.986307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f9f34-5d1d-4ce6-997d-bfb1419c1c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:37.987094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f9f34-5d1d-4ce6-997d-bfb1419c1c7e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.987472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f9f34-5d1d-4ce6-997d-bfb1419c1c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.987843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f9f34-5d1d-4ce6-997d-bfb1419c1c7e None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.989197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f9f34-5d1d-4ce6-997d-bfb1419c1c7e None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:37.990871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f9f34-5d1d-4ce6-997d-bfb1419c1c7e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:37.992754 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 116/231] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:37 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:37.999079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f9f34-5d1d-4ce6-997d-bfb1419c1c7e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:38.010957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c66e2b-0fe2-4dbd-a9f5-995df57917e4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:38.051063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a2830-3d7a-4646-a3ca-d4b12adb3520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:38.051861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a2830-3d7a-4646-a3ca-d4b12adb3520 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.052238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a2830-3d7a-4646-a3ca-d4b12adb3520 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.052644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a2830-3d7a-4646-a3ca-d4b12adb3520 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.053870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a2830-3d7a-4646-a3ca-d4b12adb3520 None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:38.055528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a2830-3d7a-4646-a3ca-d4b12adb3520 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:38.057837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 116/232] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:37 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:38.068514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a2830-3d7a-4646-a3ca-d4b12adb3520 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:38.131672 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:38.135166 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f9f34-5d1d-4ce6-997d-bfb1419c1c7e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:38.173542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f47b15-7e79-406c-a5a3-610c9c4d8f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:38.174284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f47b15-7e79-406c-a5a3-610c9c4d8f4f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.174637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f47b15-7e79-406c-a5a3-610c9c4d8f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.174983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f47b15-7e79-406c-a5a3-610c9c4d8f4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.175790 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f47b15-7e79-406c-a5a3-610c9c4d8f4f None admin] RBAC: Adding query filter params (role.id=c70d64f54bff4d2ba00749c556be9c49, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c, user.id=8a78b727aba946f9b76b64b4943b8bc5) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:38.176174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f47b15-7e79-406c-a5a3-610c9c4d8f4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:38.177725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f47b15-7e79-406c-a5a3-610c9c4d8f4f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:38.184429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 117/233] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:37 2022] GET /identity/v3/role_assignments?user.id=8a78b727aba946f9b76b64b4943b8bc5&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:38.272825 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:38.284193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a2830-3d7a-4646-a3ca-d4b12adb3520 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:38.338063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb052b-5322-4e6a-bc2e-584e9eaf95fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:38.338738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb052b-5322-4e6a-bc2e-584e9eaf95fd None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.339019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb052b-5322-4e6a-bc2e-584e9eaf95fd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.339905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb052b-5322-4e6a-bc2e-584e9eaf95fd None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/users/b227303aa9ee413c815f7c3ab94a0845/roles/6f9c4ad710e84eaabc415a1dd1541055` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.342439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb052b-5322-4e6a-bc2e-584e9eaf95fd None admin] RBAC: Authorizing `identity:create_grant(project_id=272ff39535e04ccb8b5d3e8facf8135c, user_id=b227303aa9ee413c815f7c3ab94a0845, role_id=6f9c4ad710e84eaabc415a1dd1541055)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:38.343917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb052b-5322-4e6a-bc2e-584e9eaf95fd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:38.361521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 117/234] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:29:38 2022] PUT /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/users/b227303aa9ee413c815f7c3ab94a0845/roles/6f9c4ad710e84eaabc415a1dd1541055 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:39.406615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e11dc-6cfa-45dd-b670-e334a1a9b4be None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:39.407639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e11dc-6cfa-45dd-b670-e334a1a9b4be None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:39.408149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e11dc-6cfa-45dd-b670-e334a1a9b4be None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:39.409759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 118/235] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:29:39.415340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329283d-3a84-4c21-81ed-a1009d2dde0a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:39.416141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329283d-3a84-4c21-81ed-a1009d2dde0a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:39.416659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329283d-3a84-4c21-81ed-a1009d2dde0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:39.437940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3329283d-3a84-4c21-81ed-a1009d2dde0a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:39.439705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3329283d-3a84-4c21-81ed-a1009d2dde0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:39.509975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 118/236] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:39.897839 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e11dc-6cfa-45dd-b670-e334a1a9b4be None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:40.005125 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e11dc-6cfa-45dd-b670-e334a1a9b4be None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:40.037312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeaa19-0faa-4862-8e00-05ae165ab822 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.037925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeaa19-0faa-4862-8e00-05ae165ab822 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.038459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeaa19-0faa-4862-8e00-05ae165ab822 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.040030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 119/237] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:29:40.046582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b87947-f6b6-49de-a998-202119ff146f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.047113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b87947-f6b6-49de-a998-202119ff146f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.047601 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b87947-f6b6-49de-a998-202119ff146f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.055940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee822da-218f-455d-b524-58bac40f2871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:40.056890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee822da-218f-455d-b524-58bac40f2871 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.064190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee822da-218f-455d-b524-58bac40f2871 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.064724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee822da-218f-455d-b524-58bac40f2871 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.085624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e3b87947-f6b6-49de-a998-202119ff146f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:40.087492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e3b87947-f6b6-49de-a998-202119ff146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:40.089088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee822da-218f-455d-b524-58bac40f2871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-2ee822da-218f-455d-b524-58bac40f2871 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:40.099497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:40.100410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:40.101263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 119/238] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:29:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:40.101919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fa918-a583-4343-bae6-8055cc6882eb None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.102542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fa918-a583-4343-bae6-8055cc6882eb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.102705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fa918-a583-4343-bae6-8055cc6882eb None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.104321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 120/239] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:40.110599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fa918-a583-4343-bae6-8055cc6882eb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:40.138352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:40.190713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 120/240] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:40.196680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e97047-545f-44fb-8fb5-644f5152dc21 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.196680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e97047-545f-44fb-8fb5-644f5152dc21 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.196680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e97047-545f-44fb-8fb5-644f5152dc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.231333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-86e97047-545f-44fb-8fb5-644f5152dc21 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:40.232670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-86e97047-545f-44fb-8fb5-644f5152dc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:40.246068 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fa918-a583-4343-bae6-8055cc6882eb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:40.254411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:40.322745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311884f-29cf-479e-b69a-f880bcfa81fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:40.324236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311884f-29cf-479e-b69a-f880bcfa81fa None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.324754 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311884f-29cf-479e-b69a-f880bcfa81fa None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.325206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311884f-29cf-479e-b69a-f880bcfa81fa None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.326081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311884f-29cf-479e-b69a-f880bcfa81fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:40.328158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311884f-29cf-479e-b69a-f880bcfa81fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:40.330018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311884f-29cf-479e-b69a-f880bcfa81fa None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:40.341227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 121/241] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:40.351020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311884f-29cf-479e-b69a-f880bcfa81fa None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:40.373031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 121/242] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:40.456452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:40.479325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311884f-29cf-479e-b69a-f880bcfa81fa None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:40.515701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e97047-545f-44fb-8fb5-644f5152dc21 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:40.521648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504e423-764b-46a5-9b7f-f52b0ea0ac8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:40.522523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504e423-764b-46a5-9b7f-f52b0ea0ac8f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.523021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504e423-764b-46a5-9b7f-f52b0ea0ac8f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.523492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504e423-764b-46a5-9b7f-f52b0ea0ac8f None admin] PATH_INFO: `/v3/users/b227303aa9ee413c815f7c3ab94a0845` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.525866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504e423-764b-46a5-9b7f-f52b0ea0ac8f None admin] RBAC: Authorizing `identity:get_user(user_id=b227303aa9ee413c815f7c3ab94a0845)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:40.528512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504e423-764b-46a5-9b7f-f52b0ea0ac8f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:40.531028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 122/243] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:40 2022] GET /identity/v3/users/b227303aa9ee413c815f7c3ab94a0845 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:40.537333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504e423-764b-46a5-9b7f-f52b0ea0ac8f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:40.703875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e97047-545f-44fb-8fb5-644f5152dc21 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:40.724883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504e423-764b-46a5-9b7f-f52b0ea0ac8f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:40.749841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d84ee-be75-49ff-a971-e7bd2dab073f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:40.751227 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d84ee-be75-49ff-a971-e7bd2dab073f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.751891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d84ee-be75-49ff-a971-e7bd2dab073f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.752310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d84ee-be75-49ff-a971-e7bd2dab073f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.754322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d84ee-be75-49ff-a971-e7bd2dab073f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:40.756530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d84ee-be75-49ff-a971-e7bd2dab073f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:40.758972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 122/244] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:40.760501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e988a3c-b21c-4a0b-ac0b-4a6ba11559f7 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.760869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e988a3c-b21c-4a0b-ac0b-4a6ba11559f7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.761681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e988a3c-b21c-4a0b-ac0b-4a6ba11559f7 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.767405 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 123/245] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:40.767923 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6bc71-8b64-4180-ac8c-e47433e29bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:40.768562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6bc71-8b64-4180-ac8c-e47433e29bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.768845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6bc71-8b64-4180-ac8c-e47433e29bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.769065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e988a3c-b21c-4a0b-ac0b-4a6ba11559f7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:40.769162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6bc71-8b64-4180-ac8c-e47433e29bf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.785141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:40.799589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:40.846636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6bc71-8b64-4180-ac8c-e47433e29bf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:40.848101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6bc71-8b64-4180-ac8c-e47433e29bf1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:40.881397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e988a3c-b21c-4a0b-ac0b-4a6ba11559f7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:40.914731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55539054-3067-4911-8acf-3db035359aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:40.915330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55539054-3067-4911-8acf-3db035359aa1 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.915607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55539054-3067-4911-8acf-3db035359aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.915883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55539054-3067-4911-8acf-3db035359aa1 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.916545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55539054-3067-4911-8acf-3db035359aa1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:40.918508 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55539054-3067-4911-8acf-3db035359aa1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:40.938865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 124/246] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:29:40 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:40.940105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d984be-c8b4-4b1d-9cea-bdc16edac6f9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:40.940393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d984be-c8b4-4b1d-9cea-bdc16edac6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:40.941232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d984be-c8b4-4b1d-9cea-bdc16edac6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:40.960194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-87d984be-c8b4-4b1d-9cea-bdc16edac6f9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:40.961461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-87d984be-c8b4-4b1d-9cea-bdc16edac6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:40.961990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 123/247] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 11:29:40.962779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6bc71-8b64-4180-ac8c-e47433e29bf1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:41.028400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:41.037741 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:41.048115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 125/248] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:41.049460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7ad98-b1fb-4e4b-8895-a1f72ee74bb2 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:41.049740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7ad98-b1fb-4e4b-8895-a1f72ee74bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:41.050012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7ad98-b1fb-4e4b-8895-a1f72ee74bb2 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:41.051653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 126/249] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:41.052426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7ad98-b1fb-4e4b-8895-a1f72ee74bb2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:41.084512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6bc71-8b64-4180-ac8c-e47433e29bf1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:41.132168 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab7387-fe77-431c-84d4-bda2808aa5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:41.132952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab7387-fe77-431c-84d4-bda2808aa5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:41.133298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab7387-fe77-431c-84d4-bda2808aa5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:41.133698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab7387-fe77-431c-84d4-bda2808aa5f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:41.148483 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab7387-fe77-431c-84d4-bda2808aa5f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-7bab7387-fe77-431c-84d4-bda2808aa5f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:41.154473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:41.155347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:41.156136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 124/250] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:41.161992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1621f7-e7c1-46a4-bb0a-37a7d3ebcc5b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:41.161992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1621f7-e7c1-46a4-bb0a-37a7d3ebcc5b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:41.161992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1621f7-e7c1-46a4-bb0a-37a7d3ebcc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:41.180782 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3f1621f7-e7c1-46a4-bb0a-37a7d3ebcc5b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:41.182006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3f1621f7-e7c1-46a4-bb0a-37a7d3ebcc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:41.194375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7ad98-b1fb-4e4b-8895-a1f72ee74bb2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:41.241654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0247a-a10e-4f0e-85ca-df0aed1cd8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:41.242289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0247a-a10e-4f0e-85ca-df0aed1cd8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:41.242559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0247a-a10e-4f0e-85ca-df0aed1cd8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:41.242832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0247a-a10e-4f0e-85ca-df0aed1cd8f9 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:41.243932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0247a-a10e-4f0e-85ca-df0aed1cd8f9 None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:41.245948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0247a-a10e-4f0e-85ca-df0aed1cd8f9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:41.248027 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 127/251] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:41 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:41.248851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0247a-a10e-4f0e-85ca-df0aed1cd8f9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:41.262814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 125/252] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:41.263502 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1621f7-e7c1-46a4-bb0a-37a7d3ebcc5b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:41.325317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:41.326858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:41.383357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0247a-a10e-4f0e-85ca-df0aed1cd8f9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:41.405473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1621f7-e7c1-46a4-bb0a-37a7d3ebcc5b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:41.422224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9d162-f691-4ec0-ad8f-1c61c571c56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:41.422888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9d162-f691-4ec0-ad8f-1c61c571c56b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:41.423174 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9d162-f691-4ec0-ad8f-1c61c571c56b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:41.423449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9d162-f691-4ec0-ad8f-1c61c571c56b None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:41.424120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9d162-f691-4ec0-ad8f-1c61c571c56b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:41.424423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9d162-f691-4ec0-ad8f-1c61c571c56b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:41.425884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9d162-f691-4ec0-ad8f-1c61c571c56b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:41.435430 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 128/253] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:41.436566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9d162-f691-4ec0-ad8f-1c61c571c56b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:41.452815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc8718-b792-4d5b-892f-58ed8b0f0cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:41.453351 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc8718-b792-4d5b-892f-58ed8b0f0cda None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:41.453795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc8718-b792-4d5b-892f-58ed8b0f0cda None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:41.454073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc8718-b792-4d5b-892f-58ed8b0f0cda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:41.455189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc8718-b792-4d5b-892f-58ed8b0f0cda None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c, user.id=b227303aa9ee413c815f7c3ab94a0845) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:41.455507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc8718-b792-4d5b-892f-58ed8b0f0cda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:41.457332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc8718-b792-4d5b-892f-58ed8b0f0cda None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:41.465593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 126/254] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:41 2022] GET /identity/v3/role_assignments?user.id=b227303aa9ee413c815f7c3ab94a0845&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:41.467210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc8718-b792-4d5b-892f-58ed8b0f0cda None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:41.603398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc8718-b792-4d5b-892f-58ed8b0f0cda None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:41.614757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:41.632245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9d162-f691-4ec0-ad8f-1c61c571c56b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:41.651189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39230d29-a331-45d6-9f18-6d0cf8bceace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:41.652168 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39230d29-a331-45d6-9f18-6d0cf8bceace None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:41.652449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39230d29-a331-45d6-9f18-6d0cf8bceace None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:41.653143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39230d29-a331-45d6-9f18-6d0cf8bceace None admin] PATH_INFO: `/v3/users/709947439b3c4c5a94ba7b71f00d4e83` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:41.662037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:41.672201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b1818-74e3-4f5b-9628-2b149b26f679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:41.672896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b1818-74e3-4f5b-9628-2b149b26f679 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:41.673955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39230d29-a331-45d6-9f18-6d0cf8bceace None admin] RBAC: Authorizing `identity:get_user(user_id=709947439b3c4c5a94ba7b71f00d4e83)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:41.675584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39230d29-a331-45d6-9f18-6d0cf8bceace None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:41.677787 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 127/255] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:41 2022] GET /identity/v3/users/709947439b3c4c5a94ba7b71f00d4e83 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:41.688966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b1818-74e3-4f5b-9628-2b149b26f679 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:41.689712 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b1818-74e3-4f5b-9628-2b149b26f679 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:41.691190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39230d29-a331-45d6-9f18-6d0cf8bceace None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:41.710228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b1818-74e3-4f5b-9628-2b149b26f679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-061b1818-74e3-4f5b-9628-2b149b26f679 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:41.718051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:41.719010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:41.719788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 129/256] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:29:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:41.727815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b1818-74e3-4f5b-9628-2b149b26f679 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:41.866066 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39230d29-a331-45d6-9f18-6d0cf8bceace None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:41.880384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b1818-74e3-4f5b-9628-2b149b26f679 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:41.906689 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c5730-3f7f-4000-b972-ac6e923b82d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:41.907303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c5730-3f7f-4000-b972-ac6e923b82d9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:41.907580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c5730-3f7f-4000-b972-ac6e923b82d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:41.907850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c5730-3f7f-4000-b972-ac6e923b82d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:41.930222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc5c6e-472c-42ab-a2c4-ee624ab3abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:41.930844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc5c6e-472c-42ab-a2c4-ee624ab3abe4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:41.931127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc5c6e-472c-42ab-a2c4-ee624ab3abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:41.931402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc5c6e-472c-42ab-a2c4-ee624ab3abe4 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:41.932075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc5c6e-472c-42ab-a2c4-ee624ab3abe4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:41.932380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc5c6e-472c-42ab-a2c4-ee624ab3abe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:41.934454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc5c6e-472c-42ab-a2c4-ee624ab3abe4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:41.952780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 130/257] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:29:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:41.953702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc5c6e-472c-42ab-a2c4-ee624ab3abe4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:41.959087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:41.999142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c5730-3f7f-4000-b972-ac6e923b82d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:42.000553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c5730-3f7f-4000-b972-ac6e923b82d9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:42.010399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:42.072724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc5c6e-472c-42ab-a2c4-ee624ab3abe4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:42.109162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ff024-2119-4232-98c8-bc5e572ed31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:42.109883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ff024-2119-4232-98c8-bc5e572ed31e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:42.110212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ff024-2119-4232-98c8-bc5e572ed31e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:42.110483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ff024-2119-4232-98c8-bc5e572ed31e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:42.124054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ff024-2119-4232-98c8-bc5e572ed31e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:42.124292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 128/258] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:42.129559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c5730-3f7f-4000-b972-ac6e923b82d9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-670ff024-2119-4232-98c8-bc5e572ed31e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:42.130052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:42.130899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:42.131690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 131/259] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:42.134448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ff024-2119-4232-98c8-bc5e572ed31e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:42.219880 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:42.257154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ff024-2119-4232-98c8-bc5e572ed31e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:42.266590 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c5730-3f7f-4000-b972-ac6e923b82d9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:42.267534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:42.300446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d92b4-e802-41e2-bbf6-b37be1887994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:42.301386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d92b4-e802-41e2-bbf6-b37be1887994 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:42.301671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d92b4-e802-41e2-bbf6-b37be1887994 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:42.301947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d92b4-e802-41e2-bbf6-b37be1887994 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:42.303737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d92b4-e802-41e2-bbf6-b37be1887994 None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:42.305768 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d92b4-e802-41e2-bbf6-b37be1887994 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:42.310936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c103288-0caf-4f89-9741-66e91dfa3a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:42.311570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103288-0caf-4f89-9741-66e91dfa3a28 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:42.311845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103288-0caf-4f89-9741-66e91dfa3a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:42.312118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103288-0caf-4f89-9741-66e91dfa3a28 None admin] PATH_INFO: `/v3/users/b227303aa9ee413c815f7c3ab94a0845` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:42.313281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c103288-0caf-4f89-9741-66e91dfa3a28 None admin] RBAC: Authorizing `identity:get_user(user_id=b227303aa9ee413c815f7c3ab94a0845)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:42.313584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 132/260] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:42 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:42.314726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c103288-0caf-4f89-9741-66e91dfa3a28 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:42.315053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d92b4-e802-41e2-bbf6-b37be1887994 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:42.316948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 129/261] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:42 2022] GET /identity/v3/users/b227303aa9ee413c815f7c3ab94a0845 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:42.317704 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c103288-0caf-4f89-9741-66e91dfa3a28 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:42.429032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d92b4-e802-41e2-bbf6-b37be1887994 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:42.453148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c103288-0caf-4f89-9741-66e91dfa3a28 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:42.466951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78769821-3d8d-491d-a7d0-2304819fbad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:42.467367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78769821-3d8d-491d-a7d0-2304819fbad6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:42.467560 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78769821-3d8d-491d-a7d0-2304819fbad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:42.467831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78769821-3d8d-491d-a7d0-2304819fbad6 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:42.468514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78769821-3d8d-491d-a7d0-2304819fbad6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:42.468913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78769821-3d8d-491d-a7d0-2304819fbad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:42.470569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78769821-3d8d-491d-a7d0-2304819fbad6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:42.479029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 133/262] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:42.484901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78769821-3d8d-491d-a7d0-2304819fbad6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:42.505635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3538f7d-48cf-4507-8f53-3253fd235d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:42.506942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3538f7d-48cf-4507-8f53-3253fd235d1d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:42.507568 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3538f7d-48cf-4507-8f53-3253fd235d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:42.508049 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3538f7d-48cf-4507-8f53-3253fd235d1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:42.509370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3538f7d-48cf-4507-8f53-3253fd235d1d None admin] RBAC: Adding query filter params (role.id=4febe0fde4a74359af9325b7352b75d6, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c, user.id=709947439b3c4c5a94ba7b71f00d4e83) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:42.510141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3538f7d-48cf-4507-8f53-3253fd235d1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:42.512268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3538f7d-48cf-4507-8f53-3253fd235d1d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:42.515239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:42.522489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 130/263] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:42 2022] GET /identity/v3/role_assignments?user.id=709947439b3c4c5a94ba7b71f00d4e83&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=4febe0fde4a74359af9325b7352b75d6 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:42.523856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3538f7d-48cf-4507-8f53-3253fd235d1d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:42.531594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:42.654241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78769821-3d8d-491d-a7d0-2304819fbad6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:42.711808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3538f7d-48cf-4507-8f53-3253fd235d1d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:42.724995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043defb5-8de0-4f64-9063-250590227c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:42.726002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043defb5-8de0-4f64-9063-250590227c25 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:42.727779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043defb5-8de0-4f64-9063-250590227c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:42.728353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043defb5-8de0-4f64-9063-250590227c25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:42.761618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df0a38-bc01-4f1b-8e76-aaf50b45156e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:42.762198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0a38-bc01-4f1b-8e76-aaf50b45156e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:42.762487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0a38-bc01-4f1b-8e76-aaf50b45156e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:42.762762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0a38-bc01-4f1b-8e76-aaf50b45156e None admin] PATH_INFO: `/v3/users/8a78b727aba946f9b76b64b4943b8bc5` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:42.764087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df0a38-bc01-4f1b-8e76-aaf50b45156e None admin] RBAC: Authorizing `identity:get_user(user_id=8a78b727aba946f9b76b64b4943b8bc5)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:42.772514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df0a38-bc01-4f1b-8e76-aaf50b45156e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:42.772514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 131/264] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:42 2022] GET /identity/v3/users/8a78b727aba946f9b76b64b4943b8bc5 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:42.773867 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df0a38-bc01-4f1b-8e76-aaf50b45156e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:42.865112 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043defb5-8de0-4f64-9063-250590227c25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:42.866579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043defb5-8de0-4f64-9063-250590227c25 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:42.886714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:42.922141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:42.926895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df0a38-bc01-4f1b-8e76-aaf50b45156e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:42.975045 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038e6e7-ae97-49fb-bfb3-c1c15a818229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:42.975950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038e6e7-ae97-49fb-bfb3-c1c15a818229 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:42.976390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038e6e7-ae97-49fb-bfb3-c1c15a818229 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:42.976822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038e6e7-ae97-49fb-bfb3-c1c15a818229 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:42.978303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038e6e7-ae97-49fb-bfb3-c1c15a818229 None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:42.980638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038e6e7-ae97-49fb-bfb3-c1c15a818229 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:42.983020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 132/265] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:42 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:42.989112 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 134/266] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:42.989998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038e6e7-ae97-49fb-bfb3-c1c15a818229 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:42.994342 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043defb5-8de0-4f64-9063-250590227c25 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:43.139716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038e6e7-ae97-49fb-bfb3-c1c15a818229 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:43.186383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043defb5-8de0-4f64-9063-250590227c25 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:43.213835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcf216-af06-4f18-867e-1dfa6c49f33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:43.214713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcf216-af06-4f18-867e-1dfa6c49f33b None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:43.215093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcf216-af06-4f18-867e-1dfa6c49f33b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:43.215631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcf216-af06-4f18-867e-1dfa6c49f33b None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/users/8a78b727aba946f9b76b64b4943b8bc5/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:43.222956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcf216-af06-4f18-867e-1dfa6c49f33b None admin] RBAC: Authorizing `identity:create_grant(project_id=272ff39535e04ccb8b5d3e8facf8135c, user_id=8a78b727aba946f9b76b64b4943b8bc5, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:43.223804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c60e27-5dcf-4e50-a329-321c24b59a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:43.224426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcf216-af06-4f18-867e-1dfa6c49f33b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:43.224879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c60e27-5dcf-4e50-a329-321c24b59a5b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:43.225399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c60e27-5dcf-4e50-a329-321c24b59a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:43.225861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c60e27-5dcf-4e50-a329-321c24b59a5b None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:43.226775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:43.227412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c60e27-5dcf-4e50-a329-321c24b59a5b None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:43.229158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c60e27-5dcf-4e50-a329-321c24b59a5b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:43.232098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 135/267] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:42 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:43.237835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c60e27-5dcf-4e50-a329-321c24b59a5b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:43.248932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:43.260110 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 133/268] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:29:42 2022] PUT /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/users/8a78b727aba946f9b76b64b4943b8bc5/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:43.376972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c60e27-5dcf-4e50-a329-321c24b59a5b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:43.379612 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b40544-4388-44be-bdcc-92d49a9e7d07 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:43.379907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b40544-4388-44be-bdcc-92d49a9e7d07 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:43.380189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b40544-4388-44be-bdcc-92d49a9e7d07 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:43.381706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 134/269] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:29:43.390806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac6b26-2915-4319-9031-d9864641261b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:43.390806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac6b26-2915-4319-9031-d9864641261b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:43.390806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac6b26-2915-4319-9031-d9864641261b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:43.409982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f6ac6b26-2915-4319-9031-d9864641261b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:43.411537 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f6ac6b26-2915-4319-9031-d9864641261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:43.439545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64243fc-0b6d-49c6-a30f-de9af0b38c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:43.440350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64243fc-0b6d-49c6-a30f-de9af0b38c84 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:43.441247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64243fc-0b6d-49c6-a30f-de9af0b38c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:43.441521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64243fc-0b6d-49c6-a30f-de9af0b38c84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:43.442252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64243fc-0b6d-49c6-a30f-de9af0b38c84 None admin] RBAC: Adding query filter params (role.id=6f9c4ad710e84eaabc415a1dd1541055, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c, user.id=b227303aa9ee413c815f7c3ab94a0845) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:43.442558 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64243fc-0b6d-49c6-a30f-de9af0b38c84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:43.444921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64243fc-0b6d-49c6-a30f-de9af0b38c84 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:43.455862 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 136/270] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:43 2022] GET /identity/v3/role_assignments?user.id=b227303aa9ee413c815f7c3ab94a0845&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=6f9c4ad710e84eaabc415a1dd1541055 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:43.471039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86209c32-5086-4eb8-bbe7-535a4af9ad36 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:43.471039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86209c32-5086-4eb8-bbe7-535a4af9ad36 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:43.471318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86209c32-5086-4eb8-bbe7-535a4af9ad36 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:43.472916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 137/271] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:29:43.478426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5024647-5c78-40e1-a305-03516261b0bd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:43.478721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5024647-5c78-40e1-a305-03516261b0bd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:43.478976 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5024647-5c78-40e1-a305-03516261b0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:43.497397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 135/272] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:43.508864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b5024647-5c78-40e1-a305-03516261b0bd None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:43.510100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b5024647-5c78-40e1-a305-03516261b0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:43.579779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:43.584233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 138/273] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:43.806990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac6b26-2915-4319-9031-d9864641261b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:43.899194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5024647-5c78-40e1-a305-03516261b0bd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:43.930146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac6b26-2915-4319-9031-d9864641261b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:43.962761 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:43.978152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f50ca9-3181-4fd6-9a19-e1264eb472d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:43.978791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f50ca9-3181-4fd6-9a19-e1264eb472d2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:43.979497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f50ca9-3181-4fd6-9a19-e1264eb472d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:43.980305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f50ca9-3181-4fd6-9a19-e1264eb472d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:43.995247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f50ca9-3181-4fd6-9a19-e1264eb472d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-04f50ca9-3181-4fd6-9a19-e1264eb472d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:44.002521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:44.003415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:44.004920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 136/274] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:44.009234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f50ca9-3181-4fd6-9a19-e1264eb472d2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:44.039252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5024647-5c78-40e1-a305-03516261b0bd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:44.079106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a914ac-6b8f-403c-a40c-4d847ba60b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:44.080233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a914ac-6b8f-403c-a40c-4d847ba60b8c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:44.081156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a914ac-6b8f-403c-a40c-4d847ba60b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:44.081633 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a914ac-6b8f-403c-a40c-4d847ba60b8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:44.083039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a914ac-6b8f-403c-a40c-4d847ba60b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:44.085015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a914ac-6b8f-403c-a40c-4d847ba60b8c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:44.087442 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 139/275] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:44.095830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a914ac-6b8f-403c-a40c-4d847ba60b8c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:44.136765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f50ca9-3181-4fd6-9a19-e1264eb472d2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:44.172344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840e3f8-23c0-4e86-a60f-1bc7354f873a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:44.173024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840e3f8-23c0-4e86-a60f-1bc7354f873a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:44.173303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840e3f8-23c0-4e86-a60f-1bc7354f873a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:44.173576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840e3f8-23c0-4e86-a60f-1bc7354f873a None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:44.174070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:44.174243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840e3f8-23c0-4e86-a60f-1bc7354f873a None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:44.174538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840e3f8-23c0-4e86-a60f-1bc7354f873a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:44.175911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840e3f8-23c0-4e86-a60f-1bc7354f873a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:44.184954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 137/276] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:44.191330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840e3f8-23c0-4e86-a60f-1bc7354f873a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:44.210929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a914ac-6b8f-403c-a40c-4d847ba60b8c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:44.247545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a57025-77a6-4eaf-8a64-d5de7ae6ce0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:44.248219 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a57025-77a6-4eaf-8a64-d5de7ae6ce0b None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:44.248514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a57025-77a6-4eaf-8a64-d5de7ae6ce0b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:44.248799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a57025-77a6-4eaf-8a64-d5de7ae6ce0b None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:44.249142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:44.250591 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a57025-77a6-4eaf-8a64-d5de7ae6ce0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:44.252313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a57025-77a6-4eaf-8a64-d5de7ae6ce0b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:44.279358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 140/277] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:29:44 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:29:44.316997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840e3f8-23c0-4e86-a60f-1bc7354f873a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:44.357948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a829d44-eeab-497d-bd98-e3daad6090b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:44.358589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a829d44-eeab-497d-bd98-e3daad6090b0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:44.358868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a829d44-eeab-497d-bd98-e3daad6090b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:44.359192 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a829d44-eeab-497d-bd98-e3daad6090b0 None admin] PATH_INFO: `/v3/users/81bf4671fcbe4277b9a3e1094c890f95` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:44.375611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a829d44-eeab-497d-bd98-e3daad6090b0 None admin] RBAC: Authorizing `identity:get_user(user_id=81bf4671fcbe4277b9a3e1094c890f95)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:44.377151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a829d44-eeab-497d-bd98-e3daad6090b0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:44.379366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 138/278] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:44 2022] GET /identity/v3/users/81bf4671fcbe4277b9a3e1094c890f95 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:44.385348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a57025-77a6-4eaf-8a64-d5de7ae6ce0b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:44.497320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a57025-77a6-4eaf-8a64-d5de7ae6ce0b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:44.522643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:44.533730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122a700-03d9-4410-b68d-959b5d35a3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:44.534356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122a700-03d9-4410-b68d-959b5d35a3bb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:44.534916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122a700-03d9-4410-b68d-959b5d35a3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:44.535192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122a700-03d9-4410-b68d-959b5d35a3bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:44.535931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122a700-03d9-4410-b68d-959b5d35a3bb None admin] RBAC: Adding query filter params (role.id=c70d64f54bff4d2ba00749c556be9c49, user.id=81bf4671fcbe4277b9a3e1094c890f95, scope.system=all) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:44.536241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122a700-03d9-4410-b68d-959b5d35a3bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:44.537786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122a700-03d9-4410-b68d-959b5d35a3bb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:44.545225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 141/279] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Aug 31 11:29:44 2022] GET /identity/v3/role_assignments?user.id=81bf4671fcbe4277b9a3e1094c890f95&scope.system=all&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:44.979125 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277630-0d20-4a3b-93d8-e1816ad10c20 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:44.979706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277630-0d20-4a3b-93d8-e1816ad10c20 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:44.979706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277630-0d20-4a3b-93d8-e1816ad10c20 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:44.981459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 139/280] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:44.986399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5275-b6fa-4eca-9558-3c7bbf8fb92e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:44.987222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5275-b6fa-4eca-9558-3c7bbf8fb92e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:44.987707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5275-b6fa-4eca-9558-3c7bbf8fb92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:45.009093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b1fa5275-b6fa-4eca-9558-3c7bbf8fb92e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:45.010603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b1fa5275-b6fa-4eca-9558-3c7bbf8fb92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:45.098642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 142/281] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:45.420062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efab744-6395-4fe9-a587-59ddda5b9144 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:45.420436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efab744-6395-4fe9-a587-59ddda5b9144 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:45.420634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efab744-6395-4fe9-a587-59ddda5b9144 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:45.422241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 140/282] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:45.427169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aea75d-9254-41de-9f00-964369f93233 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:45.427452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aea75d-9254-41de-9f00-964369f93233 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:45.427721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aea75d-9254-41de-9f00-964369f93233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:45.453469 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e5aea75d-9254-41de-9f00-964369f93233 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:45.454647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e5aea75d-9254-41de-9f00-964369f93233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:45.529987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efab744-6395-4fe9-a587-59ddda5b9144 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:45.552285 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 143/283] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:45.649466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efab744-6395-4fe9-a587-59ddda5b9144 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:45.653014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:45.699644 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bef6b5-79f3-447c-8852-a43b6a32e0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:45.700354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bef6b5-79f3-447c-8852-a43b6a32e0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:45.700661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bef6b5-79f3-447c-8852-a43b6a32e0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:45.700938 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bef6b5-79f3-447c-8852-a43b6a32e0a9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:45.717611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bef6b5-79f3-447c-8852-a43b6a32e0a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-06bef6b5-79f3-447c-8852-a43b6a32e0a9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:45.722569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:45.723442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:45.724879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 141/284] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:29:45.729631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aea75d-9254-41de-9f00-964369f93233 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:45.834628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:45.887064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bef6b5-79f3-447c-8852-a43b6a32e0a9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:45.909297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aea75d-9254-41de-9f00-964369f93233 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:45.978473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5c7c8-dbb3-40a2-808c-7501d94a697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:45.979005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5c7c8-dbb3-40a2-808c-7501d94a697a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:45.979283 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5c7c8-dbb3-40a2-808c-7501d94a697a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:45.979554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5c7c8-dbb3-40a2-808c-7501d94a697a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:45.980626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5c7c8-dbb3-40a2-808c-7501d94a697a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:45.980931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5c7c8-dbb3-40a2-808c-7501d94a697a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:45.982305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5c7c8-dbb3-40a2-808c-7501d94a697a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:46.000295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 144/285] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:46.024390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5c7c8-dbb3-40a2-808c-7501d94a697a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:46.089042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bef6b5-79f3-447c-8852-a43b6a32e0a9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:46.162200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f3869-f011-4446-89b7-85367ae35a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:46.163253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3869-f011-4446-89b7-85367ae35a3a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:46.163738 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3869-f011-4446-89b7-85367ae35a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:46.164220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3869-f011-4446-89b7-85367ae35a3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:46.201692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f3869-f011-4446-89b7-85367ae35a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-166f3869-f011-4446-89b7-85367ae35a3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:46.219922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:46.222283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:46.224201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 142/286] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:46.235348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f3869-f011-4446-89b7-85367ae35a3a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:46.310506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:46.375760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5c7c8-dbb3-40a2-808c-7501d94a697a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:46.409837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:46.424894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f3869-f011-4446-89b7-85367ae35a3a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:46.485572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95336b7d-42b1-48fa-8ae0-1df7285fb002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:46.486510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95336b7d-42b1-48fa-8ae0-1df7285fb002 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:46.487000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95336b7d-42b1-48fa-8ae0-1df7285fb002 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:46.487482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95336b7d-42b1-48fa-8ae0-1df7285fb002 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:46.488696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95336b7d-42b1-48fa-8ae0-1df7285fb002 None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:46.489205 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95336b7d-42b1-48fa-8ae0-1df7285fb002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:46.491881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95336b7d-42b1-48fa-8ae0-1df7285fb002 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:46.502391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632904b4-9f2e-408d-aeaa-b8a38eb3f1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:46.503384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632904b4-9f2e-408d-aeaa-b8a38eb3f1ac None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:46.503845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632904b4-9f2e-408d-aeaa-b8a38eb3f1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:46.504298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632904b4-9f2e-408d-aeaa-b8a38eb3f1ac None admin] PATH_INFO: `/v3/users/709947439b3c4c5a94ba7b71f00d4e83` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:46.506072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632904b4-9f2e-408d-aeaa-b8a38eb3f1ac None admin] RBAC: Authorizing `identity:get_user(user_id=709947439b3c4c5a94ba7b71f00d4e83)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:46.507164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 143/287] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:46.509052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632904b4-9f2e-408d-aeaa-b8a38eb3f1ac None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:46.516683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 145/288] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:46 2022] GET /identity/v3/users/709947439b3c4c5a94ba7b71f00d4e83 => generated 297 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:46.521924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95336b7d-42b1-48fa-8ae0-1df7285fb002 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:46.526140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632904b4-9f2e-408d-aeaa-b8a38eb3f1ac None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:46.686108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632904b4-9f2e-408d-aeaa-b8a38eb3f1ac None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:46.697782 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:46.718581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95336b7d-42b1-48fa-8ae0-1df7285fb002 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:46.729272 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b0d41-e9a2-47e0-aad7-0b71c7efe809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:46.730048 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b0d41-e9a2-47e0-aad7-0b71c7efe809 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:46.730289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b0d41-e9a2-47e0-aad7-0b71c7efe809 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:46.730564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b0d41-e9a2-47e0-aad7-0b71c7efe809 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:46.732093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b0d41-e9a2-47e0-aad7-0b71c7efe809 None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:46.733913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b0d41-e9a2-47e0-aad7-0b71c7efe809 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:46.740484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 146/289] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:46 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:46.740484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3e0ac-f7e6-49e4-bfe8-156916415688 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:46.740484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3e0ac-f7e6-49e4-bfe8-156916415688 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:46.740484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3e0ac-f7e6-49e4-bfe8-156916415688 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:46.740484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 147/290] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:46.745542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eceb88-303e-4ad4-9f6e-ab6c85ac3430 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:46.745831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eceb88-303e-4ad4-9f6e-ab6c85ac3430 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:46.746111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eceb88-303e-4ad4-9f6e-ab6c85ac3430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:46.767600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17e977-8b6a-4f18-87e0-98ec12a9b882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:46.768622 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17e977-8b6a-4f18-87e0-98ec12a9b882 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:46.768932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-79eceb88-303e-4ad4-9f6e-ab6c85ac3430 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:46.769299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17e977-8b6a-4f18-87e0-98ec12a9b882 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:46.769742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17e977-8b6a-4f18-87e0-98ec12a9b882 None admin] PATH_INFO: `/v3/users/8a78b727aba946f9b76b64b4943b8bc5` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:46.770323 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-79eceb88-303e-4ad4-9f6e-ab6c85ac3430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:46.771232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17e977-8b6a-4f18-87e0-98ec12a9b882 None admin] RBAC: Authorizing `identity:get_user(user_id=8a78b727aba946f9b76b64b4943b8bc5)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:46.773058 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17e977-8b6a-4f18-87e0-98ec12a9b882 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:46.775656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 144/291] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:46 2022] GET /identity/v3/users/8a78b727aba946f9b76b64b4943b8bc5 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:46.776720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17e977-8b6a-4f18-87e0-98ec12a9b882 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:46.830248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:46.856426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 148/292] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:46.857247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eceb88-303e-4ad4-9f6e-ab6c85ac3430 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:46.906063 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17e977-8b6a-4f18-87e0-98ec12a9b882 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:46.952483 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d056f83-39d6-4d2f-83cb-65f1a0eef19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:46.953987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d056f83-39d6-4d2f-83cb-65f1a0eef19a None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:46.954444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d056f83-39d6-4d2f-83cb-65f1a0eef19a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:46.955160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d056f83-39d6-4d2f-83cb-65f1a0eef19a None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/users/709947439b3c4c5a94ba7b71f00d4e83/roles/4febe0fde4a74359af9325b7352b75d6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:46.958152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d056f83-39d6-4d2f-83cb-65f1a0eef19a None admin] RBAC: Authorizing `identity:create_grant(project_id=272ff39535e04ccb8b5d3e8facf8135c, user_id=709947439b3c4c5a94ba7b71f00d4e83, role_id=4febe0fde4a74359af9325b7352b75d6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:46.960434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d056f83-39d6-4d2f-83cb-65f1a0eef19a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:46.966590 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eceb88-303e-4ad4-9f6e-ab6c85ac3430 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:46.987425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 145/293] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:29:46 2022] PUT /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/users/709947439b3c4c5a94ba7b71f00d4e83/roles/4febe0fde4a74359af9325b7352b75d6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:46.987425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a7bc4-dea1-4e62-b3d5-1e28245a97ba None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:46.987425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a7bc4-dea1-4e62-b3d5-1e28245a97ba None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:46.987425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a7bc4-dea1-4e62-b3d5-1e28245a97ba None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:46.989411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 146/294] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:46.994918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90bf87-85c8-449c-9c56-3a77933b125d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:46.995195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90bf87-85c8-449c-9c56-3a77933b125d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:46.995457 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90bf87-85c8-449c-9c56-3a77933b125d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:47.028266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2097d-4bfe-499f-a8b3-70f32688240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:47.029801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2097d-4bfe-499f-a8b3-70f32688240c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:47.030400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9a90bf87-85c8-449c-9c56-3a77933b125d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:47.030828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2097d-4bfe-499f-a8b3-70f32688240c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:47.031115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2097d-4bfe-499f-a8b3-70f32688240c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:47.033310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9a90bf87-85c8-449c-9c56-3a77933b125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:47.060714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:47.138124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:47.172953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 147/295] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:47.173812 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90bf87-85c8-449c-9c56-3a77933b125d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:47.176715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2097d-4bfe-499f-a8b3-70f32688240c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:47.178353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2097d-4bfe-499f-a8b3-70f32688240c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:47.293926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90bf87-85c8-449c-9c56-3a77933b125d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:47.312929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 149/296] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:47.314899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb1ef-9047-426a-b72d-b2884785864f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:47.315352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb1ef-9047-426a-b72d-b2884785864f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:47.315765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb1ef-9047-426a-b72d-b2884785864f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:47.321441 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 150/297] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:47.322653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9deb1ef-9047-426a-b72d-b2884785864f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:47.337218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7026a3-a335-447c-8d12-1212b404f55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:47.338107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7026a3-a335-447c-8d12-1212b404f55e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:47.338596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7026a3-a335-447c-8d12-1212b404f55e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:47.339588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7026a3-a335-447c-8d12-1212b404f55e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:47.351017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7026a3-a335-447c-8d12-1212b404f55e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-4a7026a3-a335-447c-8d12-1212b404f55e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:47.356922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:47.357998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:47.358927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:47.361110 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 148/298] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:47.362583 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24025661-a894-40c4-9942-77279e43e929 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:47.362999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24025661-a894-40c4-9942-77279e43e929 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:47.363449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24025661-a894-40c4-9942-77279e43e929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:47.372433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:47.386142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-24025661-a894-40c4-9942-77279e43e929 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:47.387641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-24025661-a894-40c4-9942-77279e43e929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:47.430708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:47.462453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9deb1ef-9047-426a-b72d-b2884785864f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:47.472582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 149/299] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:47.473896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24025661-a894-40c4-9942-77279e43e929 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:47.499361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8e63d-4546-490a-94b6-f1f06dad85ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:47.500315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8e63d-4546-490a-94b6-f1f06dad85ad None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:47.500861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8e63d-4546-490a-94b6-f1f06dad85ad None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:47.501320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8e63d-4546-490a-94b6-f1f06dad85ad None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:47.502928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8e63d-4546-490a-94b6-f1f06dad85ad None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:47.504639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8e63d-4546-490a-94b6-f1f06dad85ad None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:47.506540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 151/300] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:47 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:47.513030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8e63d-4546-490a-94b6-f1f06dad85ad None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:47.587608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24025661-a894-40c4-9942-77279e43e929 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:47.621377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8e63d-4546-490a-94b6-f1f06dad85ad None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:47.622487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa794f-bc11-4072-873e-86cc23831cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:47.623105 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa794f-bc11-4072-873e-86cc23831cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:47.623465 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa794f-bc11-4072-873e-86cc23831cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:47.623675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa794f-bc11-4072-873e-86cc23831cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:47.624372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa794f-bc11-4072-873e-86cc23831cc2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:47.624750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa794f-bc11-4072-873e-86cc23831cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:47.626122 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa794f-bc11-4072-873e-86cc23831cc2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:47.627491 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:47.636638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 150/301] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:47.641339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa794f-bc11-4072-873e-86cc23831cc2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:47.662099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979448e-cbc9-47c9-bdf1-d886d5c280a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:47.663029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979448e-cbc9-47c9-bdf1-d886d5c280a8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:47.663508 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979448e-cbc9-47c9-bdf1-d886d5c280a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:47.664049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979448e-cbc9-47c9-bdf1-d886d5c280a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:47.665417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979448e-cbc9-47c9-bdf1-d886d5c280a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:47.667763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979448e-cbc9-47c9-bdf1-d886d5c280a8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:47.669735 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 152/302] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:47.670944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979448e-cbc9-47c9-bdf1-d886d5c280a8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:47.689921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:47.748140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa794f-bc11-4072-873e-86cc23831cc2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:47.782877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f404b4c-508b-4523-b840-4bee2fa6472c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:47.783479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f404b4c-508b-4523-b840-4bee2fa6472c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:47.783756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f404b4c-508b-4523-b840-4bee2fa6472c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:47.784079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f404b4c-508b-4523-b840-4bee2fa6472c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:47.784779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f404b4c-508b-4523-b840-4bee2fa6472c None admin] RBAC: Adding query filter params (role.id=c70d64f54bff4d2ba00749c556be9c49, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c, user.id=8a78b727aba946f9b76b64b4943b8bc5) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:47.785089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f404b4c-508b-4523-b840-4bee2fa6472c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:47.786069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979448e-cbc9-47c9-bdf1-d886d5c280a8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:47.786524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f404b4c-508b-4523-b840-4bee2fa6472c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:47.793141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 151/303] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:47 2022] GET /identity/v3/role_assignments?user.id=8a78b727aba946f9b76b64b4943b8bc5&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:47.794006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f404b4c-508b-4523-b840-4bee2fa6472c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:47.841152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be96b41-bb88-4cbd-aa88-20cec5e9f4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:47.842870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be96b41-bb88-4cbd-aa88-20cec5e9f4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:47.843400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be96b41-bb88-4cbd-aa88-20cec5e9f4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:47.843899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be96b41-bb88-4cbd-aa88-20cec5e9f4c8 None admin] PATH_INFO: `/v3/users/d9c88cc8e64b4777ba73ef84363d41f0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:47.864544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be96b41-bb88-4cbd-aa88-20cec5e9f4c8 None admin] RBAC: Authorizing `identity:get_user(user_id=d9c88cc8e64b4777ba73ef84363d41f0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:47.864864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:47.867129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be96b41-bb88-4cbd-aa88-20cec5e9f4c8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:47.872676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 153/304] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:47 2022] GET /identity/v3/users/d9c88cc8e64b4777ba73ef84363d41f0 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:47.873958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be96b41-bb88-4cbd-aa88-20cec5e9f4c8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:47.920390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f404b4c-508b-4523-b840-4bee2fa6472c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:47.965930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:47.974177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19998bd-eda3-4da9-aab7-ebc32fca2476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:47.974457 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19998bd-eda3-4da9-aab7-ebc32fca2476 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:47.974744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19998bd-eda3-4da9-aab7-ebc32fca2476 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:47.975352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19998bd-eda3-4da9-aab7-ebc32fca2476 None admin] PATH_INFO: `/v3/groups` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:47.976147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19998bd-eda3-4da9-aab7-ebc32fca2476 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:47.977652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19998bd-eda3-4da9-aab7-ebc32fca2476 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:48.001504 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 152/305] 10.222.0.60 () {60 vars in 1134 bytes} [Wed Aug 31 11:29:47 2022] POST /identity/v3/groups => generated 229 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:48.002153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19998bd-eda3-4da9-aab7-ebc32fca2476 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:48.020016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be96b41-bb88-4cbd-aa88-20cec5e9f4c8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:48.076973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287ad6b-458a-4095-a1d3-5e0cf438b002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:48.077683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287ad6b-458a-4095-a1d3-5e0cf438b002 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:48.077940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287ad6b-458a-4095-a1d3-5e0cf438b002 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:48.078181 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287ad6b-458a-4095-a1d3-5e0cf438b002 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:48.096959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287ad6b-458a-4095-a1d3-5e0cf438b002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9287ad6b-458a-4095-a1d3-5e0cf438b002 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:48.102474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:48.103451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:48.104257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 154/306] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:48.112666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287ad6b-458a-4095-a1d3-5e0cf438b002 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:48.158439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19998bd-eda3-4da9-aab7-ebc32fca2476 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:48.188418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:48.204912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188c187-9aab-43a9-92af-9595f6f1f55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:48.205543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188c187-9aab-43a9-92af-9595f6f1f55d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:48.205840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188c187-9aab-43a9-92af-9595f6f1f55d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:48.206127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188c187-9aab-43a9-92af-9595f6f1f55d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:48.206891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188c187-9aab-43a9-92af-9595f6f1f55d None admin] RBAC: Adding query filter params (role.id=4febe0fde4a74359af9325b7352b75d6, user.id=d9c88cc8e64b4777ba73ef84363d41f0, scope.system=all) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:48.207217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188c187-9aab-43a9-92af-9595f6f1f55d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:48.208784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188c187-9aab-43a9-92af-9595f6f1f55d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:48.217067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 153/307] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Aug 31 11:29:48 2022] GET /identity/v3/role_assignments?user.id=d9c88cc8e64b4777ba73ef84363d41f0&scope.system=all&role.id=4febe0fde4a74359af9325b7352b75d6 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:48.240194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287ad6b-458a-4095-a1d3-5e0cf438b002 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:48.273295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:48.286307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761554cc-a9a8-4ffe-85eb-b182e391a1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:48.286947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761554cc-a9a8-4ffe-85eb-b182e391a1ba None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:48.287228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761554cc-a9a8-4ffe-85eb-b182e391a1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:48.287501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761554cc-a9a8-4ffe-85eb-b182e391a1ba None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:48.288188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761554cc-a9a8-4ffe-85eb-b182e391a1ba None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:48.288571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761554cc-a9a8-4ffe-85eb-b182e391a1ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:48.289999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761554cc-a9a8-4ffe-85eb-b182e391a1ba None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:48.304629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 155/308] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:48.306766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188c187-9aab-43a9-92af-9595f6f1f55d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:48.418029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188c187-9aab-43a9-92af-9595f6f1f55d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:48.453462 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1370b0-a8a2-41a0-b78a-11d7846e880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:48.454097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1370b0-a8a2-41a0-b78a-11d7846e880d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:48.454377 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1370b0-a8a2-41a0-b78a-11d7846e880d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:48.454651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1370b0-a8a2-41a0-b78a-11d7846e880d None admin] PATH_INFO: `/v3/users/81bf4671fcbe4277b9a3e1094c890f95` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:48.455799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1370b0-a8a2-41a0-b78a-11d7846e880d None admin] RBAC: Authorizing `identity:get_user(user_id=81bf4671fcbe4277b9a3e1094c890f95)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:48.457501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1370b0-a8a2-41a0-b78a-11d7846e880d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:48.459283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 154/309] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:48 2022] GET /identity/v3/users/81bf4671fcbe4277b9a3e1094c890f95 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:48.464984 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761554cc-a9a8-4ffe-85eb-b182e391a1ba None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:48.569188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761554cc-a9a8-4ffe-85eb-b182e391a1ba None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:48.604035 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f480c59-6f26-4800-a5e5-6d489666bf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:48.605312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f480c59-6f26-4800-a5e5-6d489666bf50 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:48.605593 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f480c59-6f26-4800-a5e5-6d489666bf50 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:48.605869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f480c59-6f26-4800-a5e5-6d489666bf50 None admin] PATH_INFO: `/v3/system/users/81bf4671fcbe4277b9a3e1094c890f95/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:48.607330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f480c59-6f26-4800-a5e5-6d489666bf50 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=81bf4671fcbe4277b9a3e1094c890f95, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:48.608742 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f480c59-6f26-4800-a5e5-6d489666bf50 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:48.621713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 156/310] 10.222.0.60 () {58 vars in 1334 bytes} [Wed Aug 31 11:29:48 2022] PUT /identity/v3/system/users/81bf4671fcbe4277b9a3e1094c890f95/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:49.091186 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5a35f-9030-48b7-9fed-7f05f87198ed None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:49.091902 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5a35f-9030-48b7-9fed-7f05f87198ed None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:49.092388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5a35f-9030-48b7-9fed-7f05f87198ed None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:49.094246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 155/311] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:49.099668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beb51d-8531-4727-ae94-7512e04eab00 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:49.100562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beb51d-8531-4727-ae94-7512e04eab00 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:49.101039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beb51d-8531-4727-ae94-7512e04eab00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:49.119853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-41beb51d-8531-4727-ae94-7512e04eab00 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:49.121612 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-41beb51d-8531-4727-ae94-7512e04eab00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:49.174163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:49.196657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 157/312] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:49.463709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5a35f-9030-48b7-9fed-7f05f87198ed None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:49.519417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:49.570635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5a35f-9030-48b7-9fed-7f05f87198ed None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:49.607057 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd020ed-0cb9-40ba-a00d-155f616438ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:49.607697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd020ed-0cb9-40ba-a00d-155f616438ee None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:49.607977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd020ed-0cb9-40ba-a00d-155f616438ee None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:49.608253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd020ed-0cb9-40ba-a00d-155f616438ee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:49.620989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd020ed-0cb9-40ba-a00d-155f616438ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-cfd020ed-0cb9-40ba-a00d-155f616438ee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:49.626069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:49.626936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:49.627715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 156/313] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:49.633299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beb51d-8531-4727-ae94-7512e04eab00 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:49.743119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beb51d-8531-4727-ae94-7512e04eab00 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:49.781257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c1c3d-81f9-4d27-8e00-e459c456f94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:49.781911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c1c3d-81f9-4d27-8e00-e459c456f94f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:49.782193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c1c3d-81f9-4d27-8e00-e459c456f94f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:49.782463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c1c3d-81f9-4d27-8e00-e459c456f94f None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:49.783133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c1c3d-81f9-4d27-8e00-e459c456f94f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:49.783433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c1c3d-81f9-4d27-8e00-e459c456f94f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:49.784843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c1c3d-81f9-4d27-8e00-e459c456f94f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:49.794595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 158/314] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:49.801942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd020ed-0cb9-40ba-a00d-155f616438ee None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:49.911314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd020ed-0cb9-40ba-a00d-155f616438ee None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:49.928840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:49.946671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395310e-63fa-45a3-95d3-82fb3f64b66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:49.947296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395310e-63fa-45a3-95d3-82fb3f64b66a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:49.947573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395310e-63fa-45a3-95d3-82fb3f64b66a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:49.947845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395310e-63fa-45a3-95d3-82fb3f64b66a None admin] PATH_INFO: `/v3/users/709947439b3c4c5a94ba7b71f00d4e83` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:49.949655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395310e-63fa-45a3-95d3-82fb3f64b66a None admin] RBAC: Authorizing `identity:get_user(user_id=709947439b3c4c5a94ba7b71f00d4e83)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:49.951104 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395310e-63fa-45a3-95d3-82fb3f64b66a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:49.953013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 157/315] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:49 2022] GET /identity/v3/users/709947439b3c4c5a94ba7b71f00d4e83 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:49.958461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c1c3d-81f9-4d27-8e00-e459c456f94f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:50.009053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:50.071249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c1c3d-81f9-4d27-8e00-e459c456f94f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:50.107254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b87ca-3eb3-4d94-94c4-59f7271d8c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:50.107864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b87ca-3eb3-4d94-94c4-59f7271d8c87 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.108100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b87ca-3eb3-4d94-94c4-59f7271d8c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.108363 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b87ca-3eb3-4d94-94c4-59f7271d8c87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.114211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27fe17-36f9-4e75-a96b-4aa152b3b510 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.114495 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27fe17-36f9-4e75-a96b-4aa152b3b510 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.114770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27fe17-36f9-4e75-a96b-4aa152b3b510 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.116352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 158/316] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:50.121264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e73012-3efe-4661-808b-10067aad9021 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.121607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e73012-3efe-4661-808b-10067aad9021 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.121886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e73012-3efe-4661-808b-10067aad9021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.140795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-14e73012-3efe-4661-808b-10067aad9021 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:50.142061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-14e73012-3efe-4661-808b-10067aad9021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:50.184151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b87ca-3eb3-4d94-94c4-59f7271d8c87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:50.186021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b87ca-3eb3-4d94-94c4-59f7271d8c87 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:50.215112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 159/317] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:50.222959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cca960-df57-490f-a3fe-80802afd658d None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.223279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cca960-df57-490f-a3fe-80802afd658d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.223564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cca960-df57-490f-a3fe-80802afd658d None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.224968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 160/318] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:50.230900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36d421-d771-4cd8-9a3a-ab9fbb710d52 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.231216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36d421-d771-4cd8-9a3a-ab9fbb710d52 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.231496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36d421-d771-4cd8-9a3a-ab9fbb710d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.239865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:50.250421 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5e36d421-d771-4cd8-9a3a-ab9fbb710d52 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:50.251615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5e36d421-d771-4cd8-9a3a-ab9fbb710d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:50.298900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 159/319] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:50.308585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b87ca-3eb3-4d94-94c4-59f7271d8c87 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:50.322903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 161/320] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:50.412417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b87ca-3eb3-4d94-94c4-59f7271d8c87 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:50.451452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af6db1-0c9e-4eb5-b8e5-cf2409ce15c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:50.452410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af6db1-0c9e-4eb5-b8e5-cf2409ce15c5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.452912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af6db1-0c9e-4eb5-b8e5-cf2409ce15c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.453402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af6db1-0c9e-4eb5-b8e5-cf2409ce15c5 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.454708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af6db1-0c9e-4eb5-b8e5-cf2409ce15c5 None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:50.456315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af6db1-0c9e-4eb5-b8e5-cf2409ce15c5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:50.458147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 160/321] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:50 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:50.464944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36d421-d771-4cd8-9a3a-ab9fbb710d52 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:50.480688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af6db1-0c9e-4eb5-b8e5-cf2409ce15c5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:50.501885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:50.515080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:50.585633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36d421-d771-4cd8-9a3a-ab9fbb710d52 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:50.605409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af6db1-0c9e-4eb5-b8e5-cf2409ce15c5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:50.625358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e1d44-26f0-41b0-926a-a77b1a0ce32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:50.625966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e1d44-26f0-41b0-926a-a77b1a0ce32a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.626250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e1d44-26f0-41b0-926a-a77b1a0ce32a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.626527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e1d44-26f0-41b0-926a-a77b1a0ce32a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.627370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e1d44-26f0-41b0-926a-a77b1a0ce32a None admin] RBAC: Adding query filter params (role.id=4febe0fde4a74359af9325b7352b75d6, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c, user.id=709947439b3c4c5a94ba7b71f00d4e83) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:50.627686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e1d44-26f0-41b0-926a-a77b1a0ce32a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:50.629380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e1d44-26f0-41b0-926a-a77b1a0ce32a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:50.639359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b5cd8-e976-4fdc-bbd1-b8a83f32dbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:50.640020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b5cd8-e976-4fdc-bbd1-b8a83f32dbeb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.640307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b5cd8-e976-4fdc-bbd1-b8a83f32dbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.640619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b5cd8-e976-4fdc-bbd1-b8a83f32dbeb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.641774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b5cd8-e976-4fdc-bbd1-b8a83f32dbeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:50.642830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 162/322] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:29:50 2022] GET /identity/v3/role_assignments?user.id=709947439b3c4c5a94ba7b71f00d4e83&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=4febe0fde4a74359af9325b7352b75d6 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:50.643233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b5cd8-e976-4fdc-bbd1-b8a83f32dbeb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:50.643842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e1d44-26f0-41b0-926a-a77b1a0ce32a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:50.644985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 161/323] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:29:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:50.653908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b5cd8-e976-4fdc-bbd1-b8a83f32dbeb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:50.759184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:50.778174 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b5cd8-e976-4fdc-bbd1-b8a83f32dbeb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:50.780420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e1d44-26f0-41b0-926a-a77b1a0ce32a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:50.783947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:50.821631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07951fe2-6a51-4c92-bb31-6646c63d12ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:50.822261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951fe2-6a51-4c92-bb31-6646c63d12ab None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.822536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951fe2-6a51-4c92-bb31-6646c63d12ab None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.822804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951fe2-6a51-4c92-bb31-6646c63d12ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.825010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d15dcf-4ed4-482b-a2a3-ca0b10695c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:50.825925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d15dcf-4ed4-482b-a2a3-ca0b10695c99 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.826206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d15dcf-4ed4-482b-a2a3-ca0b10695c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.827282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d15dcf-4ed4-482b-a2a3-ca0b10695c99 None admin] PATH_INFO: `/v3/groups` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.827282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d15dcf-4ed4-482b-a2a3-ca0b10695c99 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:50.829418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d15dcf-4ed4-482b-a2a3-ca0b10695c99 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:50.834337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07951fe2-6a51-4c92-bb31-6646c63d12ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-07951fe2-6a51-4c92-bb31-6646c63d12ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:50.840543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:50.841420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:50.842198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:50.842782 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 163/324] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:50.848400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f764f08-a4c0-4773-b39f-93332a8dc5fe None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.848400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f764f08-a4c0-4773-b39f-93332a8dc5fe None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.848400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f764f08-a4c0-4773-b39f-93332a8dc5fe None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.848400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 164/325] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:50.849935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f764f08-a4c0-4773-b39f-93332a8dc5fe None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:50.853319 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 162/326] 10.222.0.60 () {60 vars in 1134 bytes} [Wed Aug 31 11:29:50 2022] POST /identity/v3/groups => generated 226 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:50.857010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffb877-4232-4013-9e5b-09045d75d002 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:50.857303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffb877-4232-4013-9e5b-09045d75d002 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:50.857582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffb877-4232-4013-9e5b-09045d75d002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:50.884148 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ffffb877-4232-4013-9e5b-09045d75d002 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:50.885461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ffffb877-4232-4013-9e5b-09045d75d002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:50.971985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 163/327] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:50.974665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f764f08-a4c0-4773-b39f-93332a8dc5fe None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:51.016450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46095f-d666-40a9-9ef0-8cde25fa1517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:51.017383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46095f-d666-40a9-9ef0-8cde25fa1517 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:51.017664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46095f-d666-40a9-9ef0-8cde25fa1517 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:51.017937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46095f-d666-40a9-9ef0-8cde25fa1517 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:51.018610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46095f-d666-40a9-9ef0-8cde25fa1517 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:51.018902 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46095f-d666-40a9-9ef0-8cde25fa1517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:51.020290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46095f-d666-40a9-9ef0-8cde25fa1517 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:51.030346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 165/328] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:51.035927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffb877-4232-4013-9e5b-09045d75d002 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:51.114047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:51.145629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffb877-4232-4013-9e5b-09045d75d002 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:51.182270 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af1b0c-ecf8-45a3-aa07-3aae4ea3c1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:51.183475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af1b0c-ecf8-45a3-aa07-3aae4ea3c1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:51.183747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af1b0c-ecf8-45a3-aa07-3aae4ea3c1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:51.184014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af1b0c-ecf8-45a3-aa07-3aae4ea3c1b5 None admin] PATH_INFO: `/v3/users/d9c88cc8e64b4777ba73ef84363d41f0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:51.185175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af1b0c-ecf8-45a3-aa07-3aae4ea3c1b5 None admin] RBAC: Authorizing `identity:get_user(user_id=d9c88cc8e64b4777ba73ef84363d41f0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:51.186677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af1b0c-ecf8-45a3-aa07-3aae4ea3c1b5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:51.188466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 164/329] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:51 2022] GET /identity/v3/users/d9c88cc8e64b4777ba73ef84363d41f0 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:51.194285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46095f-d666-40a9-9ef0-8cde25fa1517 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:51.224216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:51.248733 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af1b0c-ecf8-45a3-aa07-3aae4ea3c1b5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:51.318615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46095f-d666-40a9-9ef0-8cde25fa1517 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:51.366254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ed2e7-dab6-4d53-ad1d-908b072b9b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:51.366833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ed2e7-dab6-4d53-ad1d-908b072b9b84 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:51.367114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ed2e7-dab6-4d53-ad1d-908b072b9b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:51.367389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ed2e7-dab6-4d53-ad1d-908b072b9b84 None admin] PATH_INFO: `/v3/system/users/d9c88cc8e64b4777ba73ef84363d41f0/roles/4febe0fde4a74359af9325b7352b75d6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:51.368850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ed2e7-dab6-4d53-ad1d-908b072b9b84 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d9c88cc8e64b4777ba73ef84363d41f0, role_id=4febe0fde4a74359af9325b7352b75d6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:51.370287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ed2e7-dab6-4d53-ad1d-908b072b9b84 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:51.382193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 166/330] 10.222.0.60 () {58 vars in 1334 bytes} [Wed Aug 31 11:29:51 2022] PUT /identity/v3/system/users/d9c88cc8e64b4777ba73ef84363d41f0/roles/4febe0fde4a74359af9325b7352b75d6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:51.387125 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af1b0c-ecf8-45a3-aa07-3aae4ea3c1b5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:51.428748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2d098-74b9-45f6-85ac-d8a1aab56737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:51.429498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2d098-74b9-45f6-85ac-d8a1aab56737 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:51.429825 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2d098-74b9-45f6-85ac-d8a1aab56737 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:51.430102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2d098-74b9-45f6-85ac-d8a1aab56737 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:51.444103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2d098-74b9-45f6-85ac-d8a1aab56737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d7c2d098-74b9-45f6-85ac-d8a1aab56737 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:51.450693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:51.451663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:51.452482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 165/331] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:29:51.458593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ed2e7-dab6-4d53-ad1d-908b072b9b84 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:51.563356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:51.574990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ed2e7-dab6-4d53-ad1d-908b072b9b84 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:51.614078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb00ad-8766-4043-84c7-061512ba7525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:51.614690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb00ad-8766-4043-84c7-061512ba7525 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:51.614971 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb00ad-8766-4043-84c7-061512ba7525 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:51.615244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb00ad-8766-4043-84c7-061512ba7525 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:51.615912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb00ad-8766-4043-84c7-061512ba7525 None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:51.616213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb00ad-8766-4043-84c7-061512ba7525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:51.617689 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb00ad-8766-4043-84c7-061512ba7525 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:51.626768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 167/332] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:51.632350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2d098-74b9-45f6-85ac-d8a1aab56737 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:51.635304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:51.752190 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2d098-74b9-45f6-85ac-d8a1aab56737 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:51.788559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107db50-9789-4fa3-8c3b-34b18d297c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:51.789589 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107db50-9789-4fa3-8c3b-34b18d297c5a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:51.790096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107db50-9789-4fa3-8c3b-34b18d297c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:51.790569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107db50-9789-4fa3-8c3b-34b18d297c5a None admin] PATH_INFO: `/v3/users/81bf4671fcbe4277b9a3e1094c890f95` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:51.791918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107db50-9789-4fa3-8c3b-34b18d297c5a None admin] RBAC: Authorizing `identity:get_user(user_id=81bf4671fcbe4277b9a3e1094c890f95)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:51.793684 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107db50-9789-4fa3-8c3b-34b18d297c5a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:51.795767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 166/333] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:51 2022] GET /identity/v3/users/81bf4671fcbe4277b9a3e1094c890f95 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:51.802081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb00ad-8766-4043-84c7-061512ba7525 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:51.923254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb00ad-8766-4043-84c7-061512ba7525 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:51.969621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f921ae-7656-4692-a291-bbed09ad973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:51.970633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f921ae-7656-4692-a291-bbed09ad973b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:51.971137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f921ae-7656-4692-a291-bbed09ad973b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:51.971609 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f921ae-7656-4692-a291-bbed09ad973b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:51.972607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f921ae-7656-4692-a291-bbed09ad973b None admin] RBAC: Adding query filter params (role.id=c70d64f54bff4d2ba00749c556be9c49, user.id=81bf4671fcbe4277b9a3e1094c890f95, scope.system=all) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:51.973145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f921ae-7656-4692-a291-bbed09ad973b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:51.974808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f921ae-7656-4692-a291-bbed09ad973b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:51.991473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 168/334] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Aug 31 11:29:51 2022] GET /identity/v3/role_assignments?user.id=81bf4671fcbe4277b9a3e1094c890f95&scope.system=all&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:53.089567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108db749-f615-4a31-ab76-c527349fed34 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:53.090653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108db749-f615-4a31-ab76-c527349fed34 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:53.091137 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108db749-f615-4a31-ab76-c527349fed34 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:53.093551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 167/335] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:53.099576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f220cea-f898-446c-9c14-befbe497308b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:53.100428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f220cea-f898-446c-9c14-befbe497308b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:53.101007 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f220cea-f898-446c-9c14-befbe497308b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:53.125447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8f220cea-f898-446c-9c14-befbe497308b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:53.126997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8f220cea-f898-446c-9c14-befbe497308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:53.130323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:53.210229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 169/336] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:53.759859 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108db749-f615-4a31-ab76-c527349fed34 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:53.860659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:53.881405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbd00b-e385-4412-a21d-df27de27d5d6 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:53.882055 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbd00b-e385-4412-a21d-df27de27d5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:53.882544 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbd00b-e385-4412-a21d-df27de27d5d6 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:53.884375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 170/337] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:53.890967 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b7acd-8d58-4b32-965c-dba3afa8ffad None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:53.891258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b7acd-8d58-4b32-965c-dba3afa8ffad None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:53.891705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b7acd-8d58-4b32-965c-dba3afa8ffad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:53.902350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108db749-f615-4a31-ab76-c527349fed34 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:53.934606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ce6b7acd-8d58-4b32-965c-dba3afa8ffad None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:53.936221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ce6b7acd-8d58-4b32-965c-dba3afa8ffad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:53.944514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29351a53-14fe-478e-ab61-b3647f6d6f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:53.944514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29351a53-14fe-478e-ab61-b3647f6d6f88 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:53.944514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29351a53-14fe-478e-ab61-b3647f6d6f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:53.944514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29351a53-14fe-478e-ab61-b3647f6d6f88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:53.965313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29351a53-14fe-478e-ab61-b3647f6d6f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-29351a53-14fe-478e-ab61-b3647f6d6f88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:53.970303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:53.971268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:53.972058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:53.973194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 168/338] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:53.985167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29351a53-14fe-478e-ab61-b3647f6d6f88 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:54.067949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 171/339] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:54.121010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc040ed-f5db-4176-92a6-03f554d9328a None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:54.121618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc040ed-f5db-4176-92a6-03f554d9328a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:54.121721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc040ed-f5db-4176-92a6-03f554d9328a None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:54.124348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 172/340] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:54.128592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffa390-25f1-49ac-831b-759093145e86 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:54.128857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffa390-25f1-49ac-831b-759093145e86 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:54.129137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffa390-25f1-49ac-831b-759093145e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:54.159875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29351a53-14fe-478e-ab61-b3647f6d6f88 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:54.170401 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-97ffa390-25f1-49ac-831b-759093145e86 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:54.170699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-97ffa390-25f1-49ac-831b-759093145e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:54.202894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:54.211216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3cda9-078d-44b8-b37c-2416ae8dce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:54.212176 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3cda9-078d-44b8-b37c-2416ae8dce55 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:54.212637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3cda9-078d-44b8-b37c-2416ae8dce55 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:54.213044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3cda9-078d-44b8-b37c-2416ae8dce55 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:54.214392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3cda9-078d-44b8-b37c-2416ae8dce55 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:54.214832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3cda9-078d-44b8-b37c-2416ae8dce55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:54.216537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3cda9-078d-44b8-b37c-2416ae8dce55 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:54.226545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 169/341] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:54.228154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09464e-4b11-4abe-bde8-94d8cc794fc1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:54.228608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09464e-4b11-4abe-bde8-94d8cc794fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:54.229074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09464e-4b11-4abe-bde8-94d8cc794fc1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:54.230865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 170/342] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:54.231879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09464e-4b11-4abe-bde8-94d8cc794fc1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:54.269701 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:54.280599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 173/343] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:54.281925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9352e-63f9-4ab4-8c1d-8f71e0d5b464 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:54.282277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9352e-63f9-4ab4-8c1d-8f71e0d5b464 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:54.282561 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9352e-63f9-4ab4-8c1d-8f71e0d5b464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:54.315586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b5b9352e-63f9-4ab4-8c1d-8f71e0d5b464 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:54.317066 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b5b9352e-63f9-4ab4-8c1d-8f71e0d5b464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:54.365349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09464e-4b11-4abe-bde8-94d8cc794fc1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:54.401511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 174/344] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:54.415661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ed32a-c081-4177-b08e-bae8a539fa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:54.416578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ed32a-c081-4177-b08e-bae8a539fa55 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:54.417583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ed32a-c081-4177-b08e-bae8a539fa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:54.418024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ed32a-c081-4177-b08e-bae8a539fa55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:54.487270 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:54.500966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ed32a-c081-4177-b08e-bae8a539fa55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:54.502597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ed32a-c081-4177-b08e-bae8a539fa55 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:54.577088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9352e-63f9-4ab4-8c1d-8f71e0d5b464 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:54.666948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 171/345] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 11:29:54.672752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ed32a-c081-4177-b08e-bae8a539fa55 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:54.741364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9352e-63f9-4ab4-8c1d-8f71e0d5b464 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:54.777269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:54.782273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af730b-f551-4887-90f5-d9ec9cb3108f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:54.782919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af730b-f551-4887-90f5-d9ec9cb3108f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:54.783237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af730b-f551-4887-90f5-d9ec9cb3108f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:54.783494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af730b-f551-4887-90f5-d9ec9cb3108f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:54.798251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af730b-f551-4887-90f5-d9ec9cb3108f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d8af730b-f551-4887-90f5-d9ec9cb3108f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:54.805116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:54.806020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:54.807002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 175/346] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:29:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:54.809088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af730b-f551-4887-90f5-d9ec9cb3108f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:54.849731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:54.862202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ed32a-c081-4177-b08e-bae8a539fa55 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:54.912793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef230a30-2ad9-45f7-ab87-0cb0f2b66b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:54.913686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef230a30-2ad9-45f7-ab87-0cb0f2b66b5a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:54.914440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef230a30-2ad9-45f7-ab87-0cb0f2b66b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:54.914723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef230a30-2ad9-45f7-ab87-0cb0f2b66b5a None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:54.916136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef230a30-2ad9-45f7-ab87-0cb0f2b66b5a None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:54.917980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef230a30-2ad9-45f7-ab87-0cb0f2b66b5a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:54.920179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 172/347] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:54 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:54.921399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef230a30-2ad9-45f7-ab87-0cb0f2b66b5a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:54.943586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af730b-f551-4887-90f5-d9ec9cb3108f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:54.988949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1a47b-efcc-4f7b-9e5a-ba3a48ac04f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:54.988949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1a47b-efcc-4f7b-9e5a-ba3a48ac04f8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:54.989206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1a47b-efcc-4f7b-9e5a-ba3a48ac04f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:54.989451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1a47b-efcc-4f7b-9e5a-ba3a48ac04f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:55.001227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1a47b-efcc-4f7b-9e5a-ba3a48ac04f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-68a1a47b-efcc-4f7b-9e5a-ba3a48ac04f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:55.006176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:55.007059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:55.007872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 176/348] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:55.012152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1a47b-efcc-4f7b-9e5a-ba3a48ac04f8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:55.079841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:55.085260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef230a30-2ad9-45f7-ab87-0cb0f2b66b5a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:55.121179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12996c1-07d7-4996-aef2-28c972d3b983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:55.121814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12996c1-07d7-4996-aef2-28c972d3b983 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:55.122093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12996c1-07d7-4996-aef2-28c972d3b983 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:55.122364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12996c1-07d7-4996-aef2-28c972d3b983 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:55.132908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12996c1-07d7-4996-aef2-28c972d3b983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f12996c1-07d7-4996-aef2-28c972d3b983 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:55.140668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:55.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:55.142397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 173/349] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:55.143036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12996c1-07d7-4996-aef2-28c972d3b983 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:55.157343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1a47b-efcc-4f7b-9e5a-ba3a48ac04f8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:55.168374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:55.200353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734336da-757d-4a58-b6e6-87d9f0813a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:55.201357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734336da-757d-4a58-b6e6-87d9f0813a96 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:55.201839 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734336da-757d-4a58-b6e6-87d9f0813a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:55.202382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734336da-757d-4a58-b6e6-87d9f0813a96 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:55.203813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734336da-757d-4a58-b6e6-87d9f0813a96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:55.204343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734336da-757d-4a58-b6e6-87d9f0813a96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:55.206080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734336da-757d-4a58-b6e6-87d9f0813a96 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:55.216464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 177/350] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:29:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:55.217744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734336da-757d-4a58-b6e6-87d9f0813a96 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:55.258588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12996c1-07d7-4996-aef2-28c972d3b983 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:55.294210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2aaa1-1feb-4ab4-9d75-d6def4fa225d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:55.294846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2aaa1-1feb-4ab4-9d75-d6def4fa225d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:55.295127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2aaa1-1feb-4ab4-9d75-d6def4fa225d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:55.295404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2aaa1-1feb-4ab4-9d75-d6def4fa225d None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:55.301968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2aaa1-1feb-4ab4-9d75-d6def4fa225d None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:55.303327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2aaa1-1feb-4ab4-9d75-d6def4fa225d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:55.305028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 174/351] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:29:55 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:55.305797 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2aaa1-1feb-4ab4-9d75-d6def4fa225d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:55.337300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734336da-757d-4a58-b6e6-87d9f0813a96 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:55.352105 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:55.383394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a6438-6614-4437-a5b9-4f4c34d89fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:55.384377 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a6438-6614-4437-a5b9-4f4c34d89fba None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:55.384896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a6438-6614-4437-a5b9-4f4c34d89fba None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:55.385367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a6438-6614-4437-a5b9-4f4c34d89fba None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:55.386706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a6438-6614-4437-a5b9-4f4c34d89fba None admin] RBAC: Adding query filter params (name=reader) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:55.387294 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a6438-6614-4437-a5b9-4f4c34d89fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:55.389035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a6438-6614-4437-a5b9-4f4c34d89fba None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:55.399470 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:55.405740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 178/352] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:55.408762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a6438-6614-4437-a5b9-4f4c34d89fba None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:55.424338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2aaa1-1feb-4ab4-9d75-d6def4fa225d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:55.461543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17a852-902f-4c21-93fa-fb0565ff2197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:55.462154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17a852-902f-4c21-93fa-fb0565ff2197 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:55.462434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17a852-902f-4c21-93fa-fb0565ff2197 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:55.462708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17a852-902f-4c21-93fa-fb0565ff2197 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:55.463379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17a852-902f-4c21-93fa-fb0565ff2197 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:55.463679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17a852-902f-4c21-93fa-fb0565ff2197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:55.466432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17a852-902f-4c21-93fa-fb0565ff2197 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:55.475292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 175/353] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:55.475901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17a852-902f-4c21-93fa-fb0565ff2197 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:55.529901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a6438-6614-4437-a5b9-4f4c34d89fba None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:55.572839 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00b901-9d8b-4a19-b707-58cead53f181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:55.575207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00b901-9d8b-4a19-b707-58cead53f181 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:55.575207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00b901-9d8b-4a19-b707-58cead53f181 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:55.575207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00b901-9d8b-4a19-b707-58cead53f181 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:55.595733 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17a852-902f-4c21-93fa-fb0565ff2197 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:55.626629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:55.644550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e758d8-a302-45a1-8fec-e1ec8b05301d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:55.644886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e758d8-a302-45a1-8fec-e1ec8b05301d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:55.645169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e758d8-a302-45a1-8fec-e1ec8b05301d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:55.645685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e758d8-a302-45a1-8fec-e1ec8b05301d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:55.646419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e758d8-a302-45a1-8fec-e1ec8b05301d None admin] RBAC: Adding query filter params (group.id=2d252c01f6974f079703843e664355e7, role.id=c70d64f54bff4d2ba00749c556be9c49, scope.project.id=3d52ac8525814206ade19cf3bc6b9271) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:55.647051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e758d8-a302-45a1-8fec-e1ec8b05301d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:55.648483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e758d8-a302-45a1-8fec-e1ec8b05301d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:55.655079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 176/354] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Aug 31 11:29:55 2022] GET /identity/v3/role_assignments?group.id=2d252c01f6974f079703843e664355e7&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:55.655983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e758d8-a302-45a1-8fec-e1ec8b05301d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:55.660826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00b901-9d8b-4a19-b707-58cead53f181 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:55.662635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00b901-9d8b-4a19-b707-58cead53f181 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:55.676841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:55.808431 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 179/355] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:55.809537 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00b901-9d8b-4a19-b707-58cead53f181 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:55.821144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e758d8-a302-45a1-8fec-e1ec8b05301d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:55.870341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b38246-7fcb-4f50-a84a-5589440a0a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:55.871116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b38246-7fcb-4f50-a84a-5589440a0a14 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:55.871392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b38246-7fcb-4f50-a84a-5589440a0a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:55.871665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b38246-7fcb-4f50-a84a-5589440a0a14 None admin] PATH_INFO: `/v3/users/d9c88cc8e64b4777ba73ef84363d41f0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:55.872971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b38246-7fcb-4f50-a84a-5589440a0a14 None admin] RBAC: Authorizing `identity:get_user(user_id=d9c88cc8e64b4777ba73ef84363d41f0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:55.874527 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b38246-7fcb-4f50-a84a-5589440a0a14 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:55.879066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 177/356] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:29:55 2022] GET /identity/v3/users/d9c88cc8e64b4777ba73ef84363d41f0 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:55.880166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2f23-06b0-4bf4-bf1d-b93253ffcffb None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:55.880454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2f23-06b0-4bf4-bf1d-b93253ffcffb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:55.881029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2f23-06b0-4bf4-bf1d-b93253ffcffb None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:55.882382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 178/357] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:55.882978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd2f23-06b0-4bf4-bf1d-b93253ffcffb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:55.915150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:55.946377 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00b901-9d8b-4a19-b707-58cead53f181 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:55.985876 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:55.996637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67334878-55b5-403e-97c8-2b455dd69444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:55.997582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334878-55b5-403e-97c8-2b455dd69444 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:55.997860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334878-55b5-403e-97c8-2b455dd69444 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:55.998132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334878-55b5-403e-97c8-2b455dd69444 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:56.016533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd2f23-06b0-4bf4-bf1d-b93253ffcffb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:56.059110 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9d6a6-302b-47b7-b8f7-d47cd9f057ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:56.060174 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9d6a6-302b-47b7-b8f7-d47cd9f057ef None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:56.060519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9d6a6-302b-47b7-b8f7-d47cd9f057ef None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:56.060810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9d6a6-302b-47b7-b8f7-d47cd9f057ef None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:56.062092 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9d6a6-302b-47b7-b8f7-d47cd9f057ef None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:56.063610 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9d6a6-302b-47b7-b8f7-d47cd9f057ef None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:56.065644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 179/358] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:55 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:56.067086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd56f80-ba52-4e89-9232-70b50c952a0f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:56.067359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd56f80-ba52-4e89-9232-70b50c952a0f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:56.067623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd56f80-ba52-4e89-9232-70b50c952a0f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:56.072965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 180/359] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:56.074488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd56f80-ba52-4e89-9232-70b50c952a0f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:56.080444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67334878-55b5-403e-97c8-2b455dd69444 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:56.081958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67334878-55b5-403e-97c8-2b455dd69444 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:56.171939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:56.220336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 180/360] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:56.221843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f731-b517-44d0-8276-809e10b4e1c8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:56.222127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f731-b517-44d0-8276-809e10b4e1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:56.222731 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f731-b517-44d0-8276-809e10b4e1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:56.236164 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd56f80-ba52-4e89-9232-70b50c952a0f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:56.246732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7a14f731-b517-44d0-8276-809e10b4e1c8 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:56.247442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7a14f731-b517-44d0-8276-809e10b4e1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:56.275485 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87aa82-5b6b-40f0-91aa-1151792dc631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:56.276095 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87aa82-5b6b-40f0-91aa-1151792dc631 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:56.277031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87aa82-5b6b-40f0-91aa-1151792dc631 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:56.277842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87aa82-5b6b-40f0-91aa-1151792dc631 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:56.278577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87aa82-5b6b-40f0-91aa-1151792dc631 None admin] RBAC: Adding query filter params (role.id=4febe0fde4a74359af9325b7352b75d6, user.id=d9c88cc8e64b4777ba73ef84363d41f0, scope.system=all) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:56.278887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87aa82-5b6b-40f0-91aa-1151792dc631 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:56.280342 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87aa82-5b6b-40f0-91aa-1151792dc631 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:56.296635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 181/361] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Aug 31 11:29:56 2022] GET /identity/v3/role_assignments?user.id=d9c88cc8e64b4777ba73ef84363d41f0&scope.system=all&role.id=4febe0fde4a74359af9325b7352b75d6 => generated 510 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:56.297343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87aa82-5b6b-40f0-91aa-1151792dc631 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:56.318565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:56.342981 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 181/362] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:56.344406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc8aaf-a707-42ce-a6cc-1d03dd8dc50b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:56.344730 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc8aaf-a707-42ce-a6cc-1d03dd8dc50b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:56.345007 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc8aaf-a707-42ce-a6cc-1d03dd8dc50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:56.367471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5bdc8aaf-a707-42ce-a6cc-1d03dd8dc50b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:56.369463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5bdc8aaf-a707-42ce-a6cc-1d03dd8dc50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:56.428905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87aa82-5b6b-40f0-91aa-1151792dc631 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:56.452982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 182/363] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:56.454182 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc8aaf-a707-42ce-a6cc-1d03dd8dc50b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:56.475968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd10a1-fcfc-4394-8570-d966b12c8037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:56.477087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd10a1-fcfc-4394-8570-d966b12c8037 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:56.477218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd10a1-fcfc-4394-8570-d966b12c8037 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:56.477413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd10a1-fcfc-4394-8570-d966b12c8037 None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:56.480954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd10a1-fcfc-4394-8570-d966b12c8037 None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:56.482420 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd10a1-fcfc-4394-8570-d966b12c8037 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:56.486582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 182/364] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:29:56 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:56.492033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd10a1-fcfc-4394-8570-d966b12c8037 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:56.528298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:56.640945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:56.665847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc8aaf-a707-42ce-a6cc-1d03dd8dc50b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:56.696306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd10a1-fcfc-4394-8570-d966b12c8037 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:56.725757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a8cb9-57eb-4cb7-aec3-17f009b1a2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:56.726652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a8cb9-57eb-4cb7-aec3-17f009b1a2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:56.726941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a8cb9-57eb-4cb7-aec3-17f009b1a2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:56.727218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a8cb9-57eb-4cb7-aec3-17f009b1a2d2 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:56.728362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a8cb9-57eb-4cb7-aec3-17f009b1a2d2 None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:56.731211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a8cb9-57eb-4cb7-aec3-17f009b1a2d2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:56.733379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 183/365] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:56 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:56.739094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a8cb9-57eb-4cb7-aec3-17f009b1a2d2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:56.770831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f30268-cae7-4c4a-8abd-a4b3641bb832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:56.771927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f30268-cae7-4c4a-8abd-a4b3641bb832 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:56.772208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f30268-cae7-4c4a-8abd-a4b3641bb832 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:56.772477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f30268-cae7-4c4a-8abd-a4b3641bb832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:56.775377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f30268-cae7-4c4a-8abd-a4b3641bb832 None admin] RBAC: Adding query filter params (group.id=2d252c01f6974f079703843e664355e7, role.id=6f9c4ad710e84eaabc415a1dd1541055, scope.project.id=3d52ac8525814206ade19cf3bc6b9271) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:56.775377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f30268-cae7-4c4a-8abd-a4b3641bb832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:56.775377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f30268-cae7-4c4a-8abd-a4b3641bb832 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:56.783100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 183/366] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Aug 31 11:29:56 2022] GET /identity/v3/role_assignments?group.id=2d252c01f6974f079703843e664355e7&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=6f9c4ad710e84eaabc415a1dd1541055 => generated 260 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:56.815252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f30268-cae7-4c4a-8abd-a4b3641bb832 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:56.891213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a8cb9-57eb-4cb7-aec3-17f009b1a2d2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:56.894501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:56.947511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ca5f4-83c0-4df2-8bbb-75bdde3727d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:56.948331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ca5f4-83c0-4df2-8bbb-75bdde3727d4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:56.950632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ca5f4-83c0-4df2-8bbb-75bdde3727d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:56.950927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ca5f4-83c0-4df2-8bbb-75bdde3727d4 None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:56.952638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ca5f4-83c0-4df2-8bbb-75bdde3727d4 None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:56.954457 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ca5f4-83c0-4df2-8bbb-75bdde3727d4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:56.957570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 184/367] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:29:56 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:56.958914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ca5f4-83c0-4df2-8bbb-75bdde3727d4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:56.965748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f30268-cae7-4c4a-8abd-a4b3641bb832 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:57.018680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0b2fb-9cf9-4482-b9d7-829f37552576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:57.019318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0b2fb-9cf9-4482-b9d7-829f37552576 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:57.019603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0b2fb-9cf9-4482-b9d7-829f37552576 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:57.019877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0b2fb-9cf9-4482-b9d7-829f37552576 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:57.022994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:57.036246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0b2fb-9cf9-4482-b9d7-829f37552576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-fdc0b2fb-9cf9-4482-b9d7-829f37552576 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:57.042842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:57.043894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 184/368] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:29:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:29:57.049381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0b2fb-9cf9-4482-b9d7-829f37552576 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:57.108886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ca5f4-83c0-4df2-8bbb-75bdde3727d4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:57.147330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb18053-0f5c-439b-852f-4001d7278a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:57.148306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb18053-0f5c-439b-852f-4001d7278a77 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:57.148836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb18053-0f5c-439b-852f-4001d7278a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:57.149316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb18053-0f5c-439b-852f-4001d7278a77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:57.162696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb18053-0f5c-439b-852f-4001d7278a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-7bb18053-0f5c-439b-852f-4001d7278a77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:57.168643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:57.169552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:29:57.170337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:57.172859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 185/369] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:29:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:57.174498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb18053-0f5c-439b-852f-4001d7278a77 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:57.178401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0b2fb-9cf9-4482-b9d7-829f37552576 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:57.214920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:57.224321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea664c8-6f3d-490d-aa98-f7cc078ae873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:57.224959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea664c8-6f3d-490d-aa98-f7cc078ae873 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:57.225236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea664c8-6f3d-490d-aa98-f7cc078ae873 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:57.225506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea664c8-6f3d-490d-aa98-f7cc078ae873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:57.226229 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea664c8-6f3d-490d-aa98-f7cc078ae873 None admin] RBAC: Adding query filter params (group.id=2d252c01f6974f079703843e664355e7, role.id=c70d64f54bff4d2ba00749c556be9c49, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:57.226536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea664c8-6f3d-490d-aa98-f7cc078ae873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:57.227968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea664c8-6f3d-490d-aa98-f7cc078ae873 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:57.235705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 185/370] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Aug 31 11:29:57 2022] GET /identity/v3/role_assignments?group.id=2d252c01f6974f079703843e664355e7&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:57.236549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea664c8-6f3d-490d-aa98-f7cc078ae873 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:57.310199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb18053-0f5c-439b-852f-4001d7278a77 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:57.356112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad61546-e478-467e-b0c4-45acd6061f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:57.357723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad61546-e478-467e-b0c4-45acd6061f59 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:57.358000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad61546-e478-467e-b0c4-45acd6061f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:57.358275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad61546-e478-467e-b0c4-45acd6061f59 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:57.358955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad61546-e478-467e-b0c4-45acd6061f59 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:57.359256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad61546-e478-467e-b0c4-45acd6061f59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:57.360683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad61546-e478-467e-b0c4-45acd6061f59 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:57.378063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:57.379797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 186/371] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:29:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:57.385415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad61546-e478-467e-b0c4-45acd6061f59 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:57.450676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea664c8-6f3d-490d-aa98-f7cc078ae873 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:57.509869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:57.522872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd11d0-9890-4363-9597-87e1d33f5d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:57.523887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd11d0-9890-4363-9597-87e1d33f5d83 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:57.530022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd11d0-9890-4363-9597-87e1d33f5d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:57.530022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd11d0-9890-4363-9597-87e1d33f5d83 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:57.530022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd11d0-9890-4363-9597-87e1d33f5d83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:57.530022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd11d0-9890-4363-9597-87e1d33f5d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:57.530022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd11d0-9890-4363-9597-87e1d33f5d83 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:57.534186 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad61546-e478-467e-b0c4-45acd6061f59 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:57.546978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 186/372] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:57.548984 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd11d0-9890-4363-9597-87e1d33f5d83 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:57.599518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ef4e5-f4db-4899-8924-e70c0f89fc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:57.600143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef4e5-f4db-4899-8924-e70c0f89fc42 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:57.600426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef4e5-f4db-4899-8924-e70c0f89fc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:57.600735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef4e5-f4db-4899-8924-e70c0f89fc42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:57.748873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ef4e5-f4db-4899-8924-e70c0f89fc42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:57.748873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ef4e5-f4db-4899-8924-e70c0f89fc42 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:57.766789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd11d0-9890-4363-9597-87e1d33f5d83 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:57.800335 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:57.817683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1e04e-ae7e-4787-a54f-a01b5c8c8daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:57.818548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1e04e-ae7e-4787-a54f-a01b5c8c8daa None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:57.818829 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1e04e-ae7e-4787-a54f-a01b5c8c8daa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:57.819100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1e04e-ae7e-4787-a54f-a01b5c8c8daa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:57.907380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1e04e-ae7e-4787-a54f-a01b5c8c8daa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:57.908861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1e04e-ae7e-4787-a54f-a01b5c8c8daa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:57.934430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:57.957146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 187/373] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 11:29:57.964734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ef4e5-f4db-4899-8924-e70c0f89fc42 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:58.055473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 187/374] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:29:58.079153 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1e04e-ae7e-4787-a54f-a01b5c8c8daa None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:58.122852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:58.170929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ef4e5-f4db-4899-8924-e70c0f89fc42 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:58.213958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23f59e-62d9-4f55-b5b9-e6c282b92952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:58.214599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23f59e-62d9-4f55-b5b9-e6c282b92952 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:58.214876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23f59e-62d9-4f55-b5b9-e6c282b92952 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:58.215161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23f59e-62d9-4f55-b5b9-e6c282b92952 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:58.216519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23f59e-62d9-4f55-b5b9-e6c282b92952 None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:58.218088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23f59e-62d9-4f55-b5b9-e6c282b92952 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:58.219763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 188/375] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:57 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:58.220963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1e04e-ae7e-4787-a54f-a01b5c8c8daa None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:58.222958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45929836-e0b0-4f1d-ba16-753604732182 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:58.223248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45929836-e0b0-4f1d-ba16-753604732182 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:58.223522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45929836-e0b0-4f1d-ba16-753604732182 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:58.229115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 189/376] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:58.229115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45929836-e0b0-4f1d-ba16-753604732182 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:58.267902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85021e6e-0a35-47ca-8fe9-dfdb9d1d7bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:58.269203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85021e6e-0a35-47ca-8fe9-dfdb9d1d7bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:58.269653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85021e6e-0a35-47ca-8fe9-dfdb9d1d7bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:58.270067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85021e6e-0a35-47ca-8fe9-dfdb9d1d7bb7 None admin] PATH_INFO: `/v3/projects/927949f45e9a418783820746376979d1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:58.271772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85021e6e-0a35-47ca-8fe9-dfdb9d1d7bb7 None admin] RBAC: Authorizing `identity:get_project(project_id=927949f45e9a418783820746376979d1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:58.273368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85021e6e-0a35-47ca-8fe9-dfdb9d1d7bb7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:58.275481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 188/377] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:58 2022] GET /identity/v3/projects/927949f45e9a418783820746376979d1 => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:58.277101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4a845-8339-49dc-87ed-f22f6569aff3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:58.277618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4a845-8339-49dc-87ed-f22f6569aff3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:58.278088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4a845-8339-49dc-87ed-f22f6569aff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:58.303835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-02a4a845-8339-49dc-87ed-f22f6569aff3 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:58.305043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-02a4a845-8339-49dc-87ed-f22f6569aff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:58.308395 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:58.366716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45929836-e0b0-4f1d-ba16-753604732182 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:58.392624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 189/378] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:58.394124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4a845-8339-49dc-87ed-f22f6569aff3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:58.406156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3d553-b3a3-4966-80a7-aad1132fce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:58.406848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3d553-b3a3-4966-80a7-aad1132fce98 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:58.407197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3d553-b3a3-4966-80a7-aad1132fce98 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:58.407580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3d553-b3a3-4966-80a7-aad1132fce98 None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:58.408852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3d553-b3a3-4966-80a7-aad1132fce98 None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:58.413330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3d553-b3a3-4966-80a7-aad1132fce98 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:58.413330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 190/379] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:29:58 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:58.420514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3d553-b3a3-4966-80a7-aad1132fce98 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:58.427368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:58.535534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4a845-8339-49dc-87ed-f22f6569aff3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:58.595832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3d553-b3a3-4966-80a7-aad1132fce98 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:58.613129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddd9f8-0a82-41c3-9225-3763b1fef890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:58.613768 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddd9f8-0a82-41c3-9225-3763b1fef890 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:58.614044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddd9f8-0a82-41c3-9225-3763b1fef890 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:58.614314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddd9f8-0a82-41c3-9225-3763b1fef890 None admin] PATH_INFO: `/v3/groups/94662cdd90d94c6ebec2ea4adacdc365` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:58.621994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddd9f8-0a82-41c3-9225-3763b1fef890 None admin] RBAC: Authorizing `identity:get_group(group_id=94662cdd90d94c6ebec2ea4adacdc365)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:58.623826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddd9f8-0a82-41c3-9225-3763b1fef890 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:58.625654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 190/380] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:29:58 2022] GET /identity/v3/groups/94662cdd90d94c6ebec2ea4adacdc365 => generated 229 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:58.631571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddd9f8-0a82-41c3-9225-3763b1fef890 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:58.637227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:58.651769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cfaad-d5a4-4ea5-b4ef-b9e38c8dea6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:58.652692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cfaad-d5a4-4ea5-b4ef-b9e38c8dea6e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:58.653059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cfaad-d5a4-4ea5-b4ef-b9e38c8dea6e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:58.653446 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cfaad-d5a4-4ea5-b4ef-b9e38c8dea6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:58.654328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cfaad-d5a4-4ea5-b4ef-b9e38c8dea6e None admin] RBAC: Adding query filter params (group.id=2d252c01f6974f079703843e664355e7, role.id=6f9c4ad710e84eaabc415a1dd1541055, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:58.654748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cfaad-d5a4-4ea5-b4ef-b9e38c8dea6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:58.656463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cfaad-d5a4-4ea5-b4ef-b9e38c8dea6e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:58.669527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 191/381] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Aug 31 11:29:58 2022] GET /identity/v3/role_assignments?group.id=2d252c01f6974f079703843e664355e7&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=6f9c4ad710e84eaabc415a1dd1541055 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:58.757060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:58.762446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddd9f8-0a82-41c3-9225-3763b1fef890 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:58.793694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cfaad-d5a4-4ea5-b4ef-b9e38c8dea6e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:58.809878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611e2ce-b8a7-4e95-ad20-cb2c97795643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:58.810601 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611e2ce-b8a7-4e95-ad20-cb2c97795643 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:58.810886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611e2ce-b8a7-4e95-ad20-cb2c97795643 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:58.811162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611e2ce-b8a7-4e95-ad20-cb2c97795643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:58.811901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611e2ce-b8a7-4e95-ad20-cb2c97795643 None admin] RBAC: Adding query filter params (group.id=94662cdd90d94c6ebec2ea4adacdc365, role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=927949f45e9a418783820746376979d1) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:58.812208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611e2ce-b8a7-4e95-ad20-cb2c97795643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:58.813710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611e2ce-b8a7-4e95-ad20-cb2c97795643 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:58.830473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 191/382] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Aug 31 11:29:58 2022] GET /identity/v3/role_assignments?group.id=94662cdd90d94c6ebec2ea4adacdc365&scope.project.id=927949f45e9a418783820746376979d1&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:58.921623 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cfaad-d5a4-4ea5-b4ef-b9e38c8dea6e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:58.964921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebeea9-e948-490b-8799-0f2d5250b198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:58.965574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebeea9-e948-490b-8799-0f2d5250b198 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:58.965857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebeea9-e948-490b-8799-0f2d5250b198 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:58.966133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebeea9-e948-490b-8799-0f2d5250b198 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:58.978119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebeea9-e948-490b-8799-0f2d5250b198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-f8ebeea9-e948-490b-8799-0f2d5250b198 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:29:58.983821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:29:58.984822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:29:58.985618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 192/383] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:29:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:29:58.991418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611e2ce-b8a7-4e95-ad20-cb2c97795643 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:59.132835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611e2ce-b8a7-4e95-ad20-cb2c97795643 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:59.172140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c38b89-b681-4571-b946-7b3ebd0fb64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:59.172716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c38b89-b681-4571-b946-7b3ebd0fb64a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:59.172999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c38b89-b681-4571-b946-7b3ebd0fb64a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:59.173273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c38b89-b681-4571-b946-7b3ebd0fb64a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:59.173938 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c38b89-b681-4571-b946-7b3ebd0fb64a None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:59.174241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c38b89-b681-4571-b946-7b3ebd0fb64a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:59.175599 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c38b89-b681-4571-b946-7b3ebd0fb64a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:59.185962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 192/384] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:29:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:59.191674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebeea9-e948-490b-8799-0f2d5250b198 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:59.280824 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:59.338633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebeea9-e948-490b-8799-0f2d5250b198 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:59.373613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f9c93-dfd7-4fbe-9ca9-0b767f112d52 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:59.374128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f9c93-dfd7-4fbe-9ca9-0b767f112d52 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:59.374128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f9c93-dfd7-4fbe-9ca9-0b767f112d52 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:59.375757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 193/385] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:29:59.382568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bcd31-95ca-422f-9f44-886243be8c90 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:59.383086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bcd31-95ca-422f-9f44-886243be8c90 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:59.383559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bcd31-95ca-422f-9f44-886243be8c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:59.395391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47456efd-d983-4bb4-9d86-af9672aa292a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:59.396039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47456efd-d983-4bb4-9d86-af9672aa292a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:59.396324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47456efd-d983-4bb4-9d86-af9672aa292a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:59.396641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47456efd-d983-4bb4-9d86-af9672aa292a None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:59.401724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47456efd-d983-4bb4-9d86-af9672aa292a None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:59.403407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47456efd-d983-4bb4-9d86-af9672aa292a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:59.405553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 193/386] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:29:59 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:59.411163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47456efd-d983-4bb4-9d86-af9672aa292a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:59.422074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ab0bcd31-95ca-422f-9f44-886243be8c90 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:59.423339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ab0bcd31-95ca-422f-9f44-886243be8c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:59.429495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:59.559046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 194/387] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:29:59.604219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47456efd-d983-4bb4-9d86-af9672aa292a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:59.636601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:59.659076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495b861-0d53-4698-925d-ac6e447899a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:59.659714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495b861-0d53-4698-925d-ac6e447899a8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:59.659996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495b861-0d53-4698-925d-ac6e447899a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:59.660271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495b861-0d53-4698-925d-ac6e447899a8 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:59.661453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495b861-0d53-4698-925d-ac6e447899a8 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:59.662888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495b861-0d53-4698-925d-ac6e447899a8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:59.664521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 194/388] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:29:59 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:29:59.671702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0bcd31-95ca-422f-9f44-886243be8c90 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:59.773798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131c75f-b987-4625-ba0c-77e29e49f1cb None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:59.774089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131c75f-b987-4625-ba0c-77e29e49f1cb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:59.774373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131c75f-b987-4625-ba0c-77e29e49f1cb None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:59.778743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 195/389] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:29:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:29:59.783941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7473c-a289-449f-b158-c2c27332ef57 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:59.784232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7473c-a289-449f-b158-c2c27332ef57 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:59.784527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7473c-a289-449f-b158-c2c27332ef57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:59.804954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b6b7473c-a289-449f-b158-c2c27332ef57 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:59.806505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b6b7473c-a289-449f-b158-c2c27332ef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:59.857207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0bcd31-95ca-422f-9f44-886243be8c90 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:59.919950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 196/390] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:29:59.921170 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7473c-a289-449f-b158-c2c27332ef57 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:59.925844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10507b80-5bf9-49d3-9481-713fc28fc641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:59.926787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10507b80-5bf9-49d3-9481-713fc28fc641 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:59.927276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10507b80-5bf9-49d3-9481-713fc28fc641 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:59.927752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10507b80-5bf9-49d3-9481-713fc28fc641 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271/groups/2d252c01f6974f079703843e664355e7/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:59.936134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10507b80-5bf9-49d3-9481-713fc28fc641 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d52ac8525814206ade19cf3bc6b9271, group_id=2d252c01f6974f079703843e664355e7, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:59.937884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10507b80-5bf9-49d3-9481-713fc28fc641 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:59.956107 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:59.961112 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 195/391] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Aug 31 11:29:59 2022] PUT /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271/groups/2d252c01f6974f079703843e664355e7/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:30:00.081849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7473c-a289-449f-b158-c2c27332ef57 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:00.106076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:00.133652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd0766-b8a5-422d-b428-b15568f55788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:00.134314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd0766-b8a5-422d-b428-b15568f55788 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:00.134598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd0766-b8a5-422d-b428-b15568f55788 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:00.134876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd0766-b8a5-422d-b428-b15568f55788 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:00.149415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd0766-b8a5-422d-b428-b15568f55788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-4afd0766-b8a5-422d-b428-b15568f55788 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:00.155238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:00.156185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:00.157038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 197/392] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:29:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:00.162259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10507b80-5bf9-49d3-9481-713fc28fc641 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:00.275387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10507b80-5bf9-49d3-9481-713fc28fc641 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:00.289833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd0766-b8a5-422d-b428-b15568f55788 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:00.319942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb6a87-b24b-46b7-8569-4027839524a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:00.320582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb6a87-b24b-46b7-8569-4027839524a3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:00.320874 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb6a87-b24b-46b7-8569-4027839524a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:00.321152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb6a87-b24b-46b7-8569-4027839524a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:00.321823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb6a87-b24b-46b7-8569-4027839524a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:00.322127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb6a87-b24b-46b7-8569-4027839524a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:00.324104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb6a87-b24b-46b7-8569-4027839524a3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:00.336814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 196/393] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:00.342546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb6a87-b24b-46b7-8569-4027839524a3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:00.408534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd0766-b8a5-422d-b428-b15568f55788 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:00.425920 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:00.450522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17365914-0f8a-4089-b380-fba92641c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:00.451523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17365914-0f8a-4089-b380-fba92641c97b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:00.452160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17365914-0f8a-4089-b380-fba92641c97b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:00.452838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17365914-0f8a-4089-b380-fba92641c97b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:00.468452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17365914-0f8a-4089-b380-fba92641c97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-17365914-0f8a-4089-b380-fba92641c97b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:00.475174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:00.476117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:30:00.476948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:00.479739 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 198/394] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:00.486180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17365914-0f8a-4089-b380-fba92641c97b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:00.492120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb6a87-b24b-46b7-8569-4027839524a3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:00.532583 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:00.545785 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ca906-f667-4fd8-93d0-e8c826d98188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:00.546433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ca906-f667-4fd8-93d0-e8c826d98188 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:00.546714 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ca906-f667-4fd8-93d0-e8c826d98188 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:00.546988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ca906-f667-4fd8-93d0-e8c826d98188 None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:00.553180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ca906-f667-4fd8-93d0-e8c826d98188 None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:00.554655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ca906-f667-4fd8-93d0-e8c826d98188 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:00.556618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 197/395] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:30:00 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:00.562416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ca906-f667-4fd8-93d0-e8c826d98188 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:00.605368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17365914-0f8a-4089-b380-fba92641c97b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:00.643528 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a6c43-bb75-456c-b637-244d24416b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:00.644880 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a6c43-bb75-456c-b637-244d24416b09 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:00.645379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a6c43-bb75-456c-b637-244d24416b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:00.645881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a6c43-bb75-456c-b637-244d24416b09 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:00.646915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a6c43-bb75-456c-b637-244d24416b09 None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:00.647425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a6c43-bb75-456c-b637-244d24416b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:00.649128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a6c43-bb75-456c-b637-244d24416b09 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:00.661033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 199/396] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:30:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:00.668395 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a6c43-bb75-456c-b637-244d24416b09 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:00.733714 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ca906-f667-4fd8-93d0-e8c826d98188 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:00.792754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:00.805806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fa6c0-8df1-4116-b138-fc803987cc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:00.806499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fa6c0-8df1-4116-b138-fc803987cc25 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:00.806783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fa6c0-8df1-4116-b138-fc803987cc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:00.807055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fa6c0-8df1-4116-b138-fc803987cc25 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:00.809900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fa6c0-8df1-4116-b138-fc803987cc25 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:00.811527 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fa6c0-8df1-4116-b138-fc803987cc25 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:00.813380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 198/397] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:30:00 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:00.819238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fa6c0-8df1-4116-b138-fc803987cc25 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:00.857255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:00.863463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a6c43-bb75-456c-b637-244d24416b09 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:00.927002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621a50f-61ce-4250-a54f-0e2b5d5abc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:00.927641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621a50f-61ce-4250-a54f-0e2b5d5abc54 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:00.927920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621a50f-61ce-4250-a54f-0e2b5d5abc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:00.928191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621a50f-61ce-4250-a54f-0e2b5d5abc54 None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:00.930571 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621a50f-61ce-4250-a54f-0e2b5d5abc54 None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:00.932011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621a50f-61ce-4250-a54f-0e2b5d5abc54 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:00.934313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 200/398] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:30:00 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:00.939979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621a50f-61ce-4250-a54f-0e2b5d5abc54 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:01.050709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fa6c0-8df1-4116-b138-fc803987cc25 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:01.055494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621a50f-61ce-4250-a54f-0e2b5d5abc54 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:01.105857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63b7f3-e8c1-454a-893a-853b6b1d1bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:01.107004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63b7f3-e8c1-454a-893a-853b6b1d1bea None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:01.107285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63b7f3-e8c1-454a-893a-853b6b1d1bea None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:01.107556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63b7f3-e8c1-454a-893a-853b6b1d1bea None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:01.109122 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63b7f3-e8c1-454a-893a-853b6b1d1bea None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:01.112167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63b7f3-e8c1-454a-893a-853b6b1d1bea None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:01.113827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 201/399] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:30:00 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:01.114475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b3d87-450c-4ac7-b378-ec8e2c6f2f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:01.115132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b3d87-450c-4ac7-b378-ec8e2c6f2f7a None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:01.115409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b3d87-450c-4ac7-b378-ec8e2c6f2f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:01.115686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b3d87-450c-4ac7-b378-ec8e2c6f2f7a None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271/groups/2d252c01f6974f079703843e664355e7/roles/6f9c4ad710e84eaabc415a1dd1541055` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:01.121615 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b3d87-450c-4ac7-b378-ec8e2c6f2f7a None admin] RBAC: Authorizing `identity:create_grant(project_id=3d52ac8525814206ade19cf3bc6b9271, group_id=2d252c01f6974f079703843e664355e7, role_id=6f9c4ad710e84eaabc415a1dd1541055)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:01.123043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b3d87-450c-4ac7-b378-ec8e2c6f2f7a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:01.128769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63b7f3-e8c1-454a-893a-853b6b1d1bea None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:01.151177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 199/400] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Aug 31 11:30:00 2022] PUT /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271/groups/2d252c01f6974f079703843e664355e7/roles/6f9c4ad710e84eaabc415a1dd1541055 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:30:01.188965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:01.256937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63b7f3-e8c1-454a-893a-853b6b1d1bea None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:01.299146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99a8a0-1fdf-4c28-9d75-222ef2969efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:01.299932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99a8a0-1fdf-4c28-9d75-222ef2969efb None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:01.300137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99a8a0-1fdf-4c28-9d75-222ef2969efb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:01.300416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99a8a0-1fdf-4c28-9d75-222ef2969efb None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/groups/2d252c01f6974f079703843e664355e7/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:01.302592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99a8a0-1fdf-4c28-9d75-222ef2969efb None admin] RBAC: Authorizing `identity:create_grant(project_id=272ff39535e04ccb8b5d3e8facf8135c, group_id=2d252c01f6974f079703843e664355e7, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:01.304013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99a8a0-1fdf-4c28-9d75-222ef2969efb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:01.320169 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 202/401] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Aug 31 11:30:01 2022] PUT /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/groups/2d252c01f6974f079703843e664355e7/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:30:01.425457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437391e-0a10-4f0f-a322-0956551f262a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:01.425804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437391e-0a10-4f0f-a322-0956551f262a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:01.426012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437391e-0a10-4f0f-a322-0956551f262a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:01.427671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 200/402] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:01.433690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ec2e1-590d-45ce-b100-2607f4bbd59e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:01.433993 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ec2e1-590d-45ce-b100-2607f4bbd59e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:01.434241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ec2e1-590d-45ce-b100-2607f4bbd59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:01.455071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-661ec2e1-590d-45ce-b100-2607f4bbd59e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:01.456409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-661ec2e1-590d-45ce-b100-2607f4bbd59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:01.548609 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 203/403] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:01.778478 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb15954-707e-45c9-83e8-4f1f442116d4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:01.778796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb15954-707e-45c9-83e8-4f1f442116d4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:01.779030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb15954-707e-45c9-83e8-4f1f442116d4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:01.780905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 201/404] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:01.786149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a612e9-aee2-4948-82c5-b64c7294c89a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:01.786486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a612e9-aee2-4948-82c5-b64c7294c89a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:01.786769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a612e9-aee2-4948-82c5-b64c7294c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:01.810443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c4a612e9-aee2-4948-82c5-b64c7294c89a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:01.811645 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c4a612e9-aee2-4948-82c5-b64c7294c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:01.836557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:01.837423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb15954-707e-45c9-83e8-4f1f442116d4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:01.890805 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 204/405] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:01.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:01.989082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb15954-707e-45c9-83e8-4f1f442116d4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:02.030313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268ce79-18a9-4013-acae-588a1b48b5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:02.030941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268ce79-18a9-4013-acae-588a1b48b5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:02.031218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268ce79-18a9-4013-acae-588a1b48b5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:02.031498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268ce79-18a9-4013-acae-588a1b48b5a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:02.048247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268ce79-18a9-4013-acae-588a1b48b5a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f268ce79-18a9-4013-acae-588a1b48b5a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:02.053214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:02.054150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:02.054989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 202/406] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:30:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:02.060229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a612e9-aee2-4948-82c5-b64c7294c89a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:02.178595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268ce79-18a9-4013-acae-588a1b48b5a1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:02.193896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a612e9-aee2-4948-82c5-b64c7294c89a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:02.241873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de77bbf-12a9-4e44-a6e9-7f791c2513a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:02.242732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de77bbf-12a9-4e44-a6e9-7f791c2513a4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:02.243159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de77bbf-12a9-4e44-a6e9-7f791c2513a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:02.243585 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de77bbf-12a9-4e44-a6e9-7f791c2513a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:02.244408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de77bbf-12a9-4e44-a6e9-7f791c2513a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:02.245434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de77bbf-12a9-4e44-a6e9-7f791c2513a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:02.249145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de77bbf-12a9-4e44-a6e9-7f791c2513a4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:02.266452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 205/407] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:02.272315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de77bbf-12a9-4e44-a6e9-7f791c2513a4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:02.312905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:02.361117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268ce79-18a9-4013-acae-588a1b48b5a1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:02.400453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:02.422672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b3006-d5e6-4653-a20a-ad45736e56a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:02.423649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b3006-d5e6-4653-a20a-ad45736e56a1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:02.424142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b3006-d5e6-4653-a20a-ad45736e56a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:02.424657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b3006-d5e6-4653-a20a-ad45736e56a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:02.446344 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b3006-d5e6-4653-a20a-ad45736e56a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a39b3006-d5e6-4653-a20a-ad45736e56a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:02.453701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:02.454623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:02.455416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:02.466402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 203/408] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:02.472224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b3006-d5e6-4653-a20a-ad45736e56a1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:02.494931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de77bbf-12a9-4e44-a6e9-7f791c2513a4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:02.563498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409381d-6670-4bc5-8a43-03856891aec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:02.565270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409381d-6670-4bc5-8a43-03856891aec3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:02.566665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409381d-6670-4bc5-8a43-03856891aec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:02.569531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409381d-6670-4bc5-8a43-03856891aec3 None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:02.570975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409381d-6670-4bc5-8a43-03856891aec3 None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:02.572663 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409381d-6670-4bc5-8a43-03856891aec3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:02.577950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 206/409] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:30:02 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:02.583361 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409381d-6670-4bc5-8a43-03856891aec3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:02.674455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b3006-d5e6-4653-a20a-ad45736e56a1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:02.719708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f947d4-e4bd-4fbe-b7b7-cddbfbd57dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:02.720367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f947d4-e4bd-4fbe-b7b7-cddbfbd57dff None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:02.720692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f947d4-e4bd-4fbe-b7b7-cddbfbd57dff None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:02.720998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f947d4-e4bd-4fbe-b7b7-cddbfbd57dff None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:02.722152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f947d4-e4bd-4fbe-b7b7-cddbfbd57dff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:02.722453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f947d4-e4bd-4fbe-b7b7-cddbfbd57dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:02.724349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f947d4-e4bd-4fbe-b7b7-cddbfbd57dff None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:02.725795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:02.737267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 204/410] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:02.738894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b4d7c-cc31-48d3-b88d-6afec4c46d56 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:02.739387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b4d7c-cc31-48d3-b88d-6afec4c46d56 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:02.739851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b4d7c-cc31-48d3-b88d-6afec4c46d56 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:02.745461 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409381d-6670-4bc5-8a43-03856891aec3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:02.745817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 205/411] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:02.746802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b4d7c-cc31-48d3-b88d-6afec4c46d56 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:02.780998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac087aec-4c96-427d-b1ac-05eee075048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:02.781626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac087aec-4c96-427d-b1ac-05eee075048e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:02.781904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac087aec-4c96-427d-b1ac-05eee075048e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:02.782173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac087aec-4c96-427d-b1ac-05eee075048e None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:02.783627 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac087aec-4c96-427d-b1ac-05eee075048e None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:02.785113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac087aec-4c96-427d-b1ac-05eee075048e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:02.791333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 207/412] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:30:02 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:02.791333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa75e-86dd-4012-85f9-b508cb01d230 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:02.791333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa75e-86dd-4012-85f9-b508cb01d230 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:02.791333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa75e-86dd-4012-85f9-b508cb01d230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:02.799300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:02.808120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c37aa75e-86dd-4012-85f9-b508cb01d230 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:02.809500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c37aa75e-86dd-4012-85f9-b508cb01d230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:02.937182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b4d7c-cc31-48d3-b88d-6afec4c46d56 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:02.960634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 208/413] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:02.961810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37aa75e-86dd-4012-85f9-b508cb01d230 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:03.037318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c37ad8-b288-4154-bf28-ff032b508e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:03.038314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37ad8-b288-4154-bf28-ff032b508e20 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.038808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37ad8-b288-4154-bf28-ff032b508e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.039284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37ad8-b288-4154-bf28-ff032b508e20 None admin] PATH_INFO: `/v3/groups/94662cdd90d94c6ebec2ea4adacdc365` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.045179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c37ad8-b288-4154-bf28-ff032b508e20 None admin] RBAC: Authorizing `identity:get_group(group_id=94662cdd90d94c6ebec2ea4adacdc365)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:03.046889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c37ad8-b288-4154-bf28-ff032b508e20 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:03.048756 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 206/414] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:30:02 2022] GET /identity/v3/groups/94662cdd90d94c6ebec2ea4adacdc365 => generated 229 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:03.054755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c37ad8-b288-4154-bf28-ff032b508e20 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:03.069858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:03.096939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37aa75e-86dd-4012-85f9-b508cb01d230 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:03.154860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05660855-71fb-468d-a547-934c86b0b242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:03.155735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05660855-71fb-468d-a547-934c86b0b242 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.156020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05660855-71fb-468d-a547-934c86b0b242 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.156292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05660855-71fb-468d-a547-934c86b0b242 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/groups/2d252c01f6974f079703843e664355e7/roles/6f9c4ad710e84eaabc415a1dd1541055` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.162622 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05660855-71fb-468d-a547-934c86b0b242 None admin] RBAC: Authorizing `identity:create_grant(project_id=272ff39535e04ccb8b5d3e8facf8135c, group_id=2d252c01f6974f079703843e664355e7, role_id=6f9c4ad710e84eaabc415a1dd1541055)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:03.164089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05660855-71fb-468d-a547-934c86b0b242 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:03.209783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 209/415] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Aug 31 11:30:02 2022] PUT /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c/groups/2d252c01f6974f079703843e664355e7/roles/6f9c4ad710e84eaabc415a1dd1541055 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:30:03.253216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c37ad8-b288-4154-bf28-ff032b508e20 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:03.265427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:03.296443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a41e90-f908-471b-be9c-c7676a1b76c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:03.300698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a41e90-f908-471b-be9c-c7676a1b76c2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.300698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a41e90-f908-471b-be9c-c7676a1b76c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.300698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a41e90-f908-471b-be9c-c7676a1b76c2 None admin] PATH_INFO: `/v3/projects/927949f45e9a418783820746376979d1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.300698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a41e90-f908-471b-be9c-c7676a1b76c2 None admin] RBAC: Authorizing `identity:get_project(project_id=927949f45e9a418783820746376979d1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:03.300698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a41e90-f908-471b-be9c-c7676a1b76c2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:03.304577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 207/416] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:30:03 2022] GET /identity/v3/projects/927949f45e9a418783820746376979d1 => generated 344 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:03.309836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05660855-71fb-468d-a547-934c86b0b242 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:03.362014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a41e90-f908-471b-be9c-c7676a1b76c2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:03.490463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05660855-71fb-468d-a547-934c86b0b242 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:03.499384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a41e90-f908-471b-be9c-c7676a1b76c2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:03.533263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900aefd-8ec6-48cc-8729-4afaeb948da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:03.534939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900aefd-8ec6-48cc-8729-4afaeb948da6 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.535427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900aefd-8ec6-48cc-8729-4afaeb948da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.536112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900aefd-8ec6-48cc-8729-4afaeb948da6 None admin] PATH_INFO: `/v3/projects/927949f45e9a418783820746376979d1/groups/94662cdd90d94c6ebec2ea4adacdc365/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.539012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900aefd-8ec6-48cc-8729-4afaeb948da6 None admin] RBAC: Authorizing `identity:create_grant(project_id=927949f45e9a418783820746376979d1, group_id=94662cdd90d94c6ebec2ea4adacdc365, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:03.540870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900aefd-8ec6-48cc-8729-4afaeb948da6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:03.541333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429b9e8-4b5b-40bb-8c65-144c0e4287c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:03.542583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429b9e8-4b5b-40bb-8c65-144c0e4287c3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.542865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429b9e8-4b5b-40bb-8c65-144c0e4287c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.543136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429b9e8-4b5b-40bb-8c65-144c0e4287c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.554625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429b9e8-4b5b-40bb-8c65-144c0e4287c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:03.560391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 210/417] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Aug 31 11:30:03 2022] PUT /identity/v3/projects/927949f45e9a418783820746376979d1/groups/94662cdd90d94c6ebec2ea4adacdc365/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a429b9e8-4b5b-40bb-8c65-144c0e4287c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:03.560545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:03.561498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:03.562273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 208/418] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:03.562935 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c998d-0c46-48b1-9484-79813b6b0412 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.568418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c998d-0c46-48b1-9484-79813b6b0412 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.568418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c998d-0c46-48b1-9484-79813b6b0412 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.568418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 209/419] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:03.570288 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c998d-0c46-48b1-9484-79813b6b0412 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:03.571324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159555f1-178c-462d-a80d-ff471f96151d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.571914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159555f1-178c-462d-a80d-ff471f96151d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.572384 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159555f1-178c-462d-a80d-ff471f96151d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.583106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:03.601610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-159555f1-178c-462d-a80d-ff471f96151d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:03.603984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-159555f1-178c-462d-a80d-ff471f96151d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:03.617134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:03.734504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c998d-0c46-48b1-9484-79813b6b0412 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:03.738436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 211/420] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:30:03.739735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819ef9b-67e4-472c-a55c-0770a27bbc7f None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.740018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819ef9b-67e4-472c-a55c-0770a27bbc7f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.745275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819ef9b-67e4-472c-a55c-0770a27bbc7f None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.747054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 212/421] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:03.753931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e66e14-94a1-4bad-b90f-03ec61c52cfc None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.754213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e66e14-94a1-4bad-b90f-03ec61c52cfc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.754486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e66e14-94a1-4bad-b90f-03ec61c52cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.779236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da558-c848-4920-bd7e-292c89e34be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:03.779843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da558-c848-4920-bd7e-292c89e34be0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.780166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da558-c848-4920-bd7e-292c89e34be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.780439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da558-c848-4920-bd7e-292c89e34be0 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.781124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8da558-c848-4920-bd7e-292c89e34be0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:03.781422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8da558-c848-4920-bd7e-292c89e34be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:03.782858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8da558-c848-4920-bd7e-292c89e34be0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:03.789588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-55e66e14-94a1-4bad-b90f-03ec61c52cfc None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:03.790805 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-55e66e14-94a1-4bad-b90f-03ec61c52cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:03.792442 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 210/422] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:30:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:03.798838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da558-c848-4920-bd7e-292c89e34be0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:03.904419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da558-c848-4920-bd7e-292c89e34be0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:03.911878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:03.951891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c477a3c-6610-46cc-b733-c869d15b0f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:03.952578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c477a3c-6610-46cc-b733-c869d15b0f6c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.952858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c477a3c-6610-46cc-b733-c869d15b0f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.953125 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c477a3c-6610-46cc-b733-c869d15b0f6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.974045 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 213/423] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:30:04.035958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c477a3c-6610-46cc-b733-c869d15b0f6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:04.037521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c477a3c-6610-46cc-b733-c869d15b0f6c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:04.089057 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e66e14-94a1-4bad-b90f-03ec61c52cfc None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:04.178661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:04.178959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 211/424] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:04.184032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:04.185365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c477a3c-6610-46cc-b733-c869d15b0f6c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:04.260322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e66e14-94a1-4bad-b90f-03ec61c52cfc None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:04.303570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c477a3c-6610-46cc-b733-c869d15b0f6c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:04.334712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f4dfd-6152-46fe-831a-94e87182bd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:04.335345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f4dfd-6152-46fe-831a-94e87182bd78 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:04.335624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f4dfd-6152-46fe-831a-94e87182bd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:04.335894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f4dfd-6152-46fe-831a-94e87182bd78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:04.351966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f4dfd-6152-46fe-831a-94e87182bd78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:04.353577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38f5e9-c61c-49dd-abe2-34febabe2501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:04.355203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38f5e9-c61c-49dd-abe2-34febabe2501 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:04.355707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38f5e9-c61c-49dd-abe2-34febabe2501 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:04.356180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38f5e9-c61c-49dd-abe2-34febabe2501 None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-dc3f4dfd-6152-46fe-831a-94e87182bd78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:04.357486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:04.358359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:04.359151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 214/425] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:04.360286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f4dfd-6152-46fe-831a-94e87182bd78 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:04.360574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38f5e9-c61c-49dd-abe2-34febabe2501 None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:04.362334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38f5e9-c61c-49dd-abe2-34febabe2501 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:04.365331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 212/426] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:30:04 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:04.369255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38f5e9-c61c-49dd-abe2-34febabe2501 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:04.464949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:04.485438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f4dfd-6152-46fe-831a-94e87182bd78 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:04.513330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38f5e9-c61c-49dd-abe2-34febabe2501 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:04.521300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:04.537284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6d00e-ef42-4ef2-9659-8d85248d81ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:04.537918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6d00e-ef42-4ef2-9659-8d85248d81ae None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:04.538217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6d00e-ef42-4ef2-9659-8d85248d81ae None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:04.538701 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6d00e-ef42-4ef2-9659-8d85248d81ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:04.551388 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b1871-1ec9-47b4-af59-6377efcd9217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:04.552180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b1871-1ec9-47b4-af59-6377efcd9217 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:04.552575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b1871-1ec9-47b4-af59-6377efcd9217 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:04.552923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b1871-1ec9-47b4-af59-6377efcd9217 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:04.553050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6d00e-ef42-4ef2-9659-8d85248d81ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:04.553716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b1871-1ec9-47b4-af59-6377efcd9217 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:04.554106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b1871-1ec9-47b4-af59-6377efcd9217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:04.555544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b1871-1ec9-47b4-af59-6377efcd9217 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-29a6d00e-ef42-4ef2-9659-8d85248d81ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:04.559382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:04.560468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:30:04.561478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:04.563605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 215/427] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:30:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:04.564845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6d00e-ef42-4ef2-9659-8d85248d81ae None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:04.568187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 213/428] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:30:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:04.569147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b1871-1ec9-47b4-af59-6377efcd9217 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:04.701764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b1871-1ec9-47b4-af59-6377efcd9217 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:04.707768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6d00e-ef42-4ef2-9659-8d85248d81ae None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:04.738906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:04.745930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813c01d-6443-4b08-96e9-157f53e45696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:04.746565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813c01d-6443-4b08-96e9-157f53e45696 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:04.746845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813c01d-6443-4b08-96e9-157f53e45696 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:04.747118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813c01d-6443-4b08-96e9-157f53e45696 None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:04.748253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813c01d-6443-4b08-96e9-157f53e45696 None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:04.749893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813c01d-6443-4b08-96e9-157f53e45696 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:04.750448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498506c-2b59-4f23-85bd-25c6fe3e2c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:04.751262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498506c-2b59-4f23-85bd-25c6fe3e2c0a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:04.751784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 216/429] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:30:04 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:04.752025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498506c-2b59-4f23-85bd-25c6fe3e2c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:04.752600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813c01d-6443-4b08-96e9-157f53e45696 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:04.752781 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498506c-2b59-4f23-85bd-25c6fe3e2c0a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:04.753624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498506c-2b59-4f23-85bd-25c6fe3e2c0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:04.754177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498506c-2b59-4f23-85bd-25c6fe3e2c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:04.755900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498506c-2b59-4f23-85bd-25c6fe3e2c0a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:04.768994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 214/430] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:04.770965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498506c-2b59-4f23-85bd-25c6fe3e2c0a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:04.817697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:04.867924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813c01d-6443-4b08-96e9-157f53e45696 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:04.894662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498506c-2b59-4f23-85bd-25c6fe3e2c0a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:04.905126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7078dc5-9498-45db-86a4-8a4435e0b4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:04.905998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7078dc5-9498-45db-86a4-8a4435e0b4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:04.906115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7078dc5-9498-45db-86a4-8a4435e0b4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:04.906419 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7078dc5-9498-45db-86a4-8a4435e0b4f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:04.932951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f514a-c6c6-418d-b28f-5e4b3d29fb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:04.933910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f514a-c6c6-418d-b28f-5e4b3d29fb92 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:04.934404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f514a-c6c6-418d-b28f-5e4b3d29fb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:04.934965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f514a-c6c6-418d-b28f-5e4b3d29fb92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:04.935989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f514a-c6c6-418d-b28f-5e4b3d29fb92 None admin] RBAC: Adding query filter params (group.id=2d252c01f6974f079703843e664355e7, role.id=c70d64f54bff4d2ba00749c556be9c49, scope.project.id=3d52ac8525814206ade19cf3bc6b9271) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:04.936506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f514a-c6c6-418d-b28f-5e4b3d29fb92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:04.938217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f514a-c6c6-418d-b28f-5e4b3d29fb92 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:04.944950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 215/431] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Aug 31 11:30:04 2022] GET /identity/v3/role_assignments?group.id=2d252c01f6974f079703843e664355e7&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:04.946098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f514a-c6c6-418d-b28f-5e4b3d29fb92 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:04.980830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7078dc5-9498-45db-86a4-8a4435e0b4f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:04.982354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7078dc5-9498-45db-86a4-8a4435e0b4f6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:05.010238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:05.040861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:05.063608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f514a-c6c6-418d-b28f-5e4b3d29fb92 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:05.097922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 217/432] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:05.105833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7078dc5-9498-45db-86a4-8a4435e0b4f6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:05.113320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57535e4-d494-48ad-86b3-bf69b7392457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:05.114402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57535e4-d494-48ad-86b3-bf69b7392457 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:05.114886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57535e4-d494-48ad-86b3-bf69b7392457 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:05.115425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57535e4-d494-48ad-86b3-bf69b7392457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:05.199404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57535e4-d494-48ad-86b3-bf69b7392457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:05.201115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57535e4-d494-48ad-86b3-bf69b7392457 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:05.227318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7078dc5-9498-45db-86a4-8a4435e0b4f6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:05.272232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23628dcc-a2a9-48aa-9b11-fe71c76d31fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:05.272828 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23628dcc-a2a9-48aa-9b11-fe71c76d31fe None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:05.273226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23628dcc-a2a9-48aa-9b11-fe71c76d31fe None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:05.273507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23628dcc-a2a9-48aa-9b11-fe71c76d31fe None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:05.274635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23628dcc-a2a9-48aa-9b11-fe71c76d31fe None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:05.277660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:05.278126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23628dcc-a2a9-48aa-9b11-fe71c76d31fe None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:05.282961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 218/433] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:30:05 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:05.288432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23628dcc-a2a9-48aa-9b11-fe71c76d31fe None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:05.313528 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:05.335976 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 216/434] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:05.342493 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57535e4-d494-48ad-86b3-bf69b7392457 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:05.397391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23628dcc-a2a9-48aa-9b11-fe71c76d31fe None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:05.435423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f8fd2-fd0a-41e4-8424-1c372111e164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:05.436557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f8fd2-fd0a-41e4-8424-1c372111e164 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:05.437031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f8fd2-fd0a-41e4-8424-1c372111e164 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:05.437492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f8fd2-fd0a-41e4-8424-1c372111e164 None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:05.438830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f8fd2-fd0a-41e4-8424-1c372111e164 None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:05.440477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f8fd2-fd0a-41e4-8424-1c372111e164 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:05.442860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 219/435] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:30:05 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:05.448155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f8fd2-fd0a-41e4-8424-1c372111e164 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:05.457031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57535e4-d494-48ad-86b3-bf69b7392457 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:05.505334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38060b-13c7-4167-8eab-1f763c111c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:05.505982 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38060b-13c7-4167-8eab-1f763c111c1b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:05.506266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38060b-13c7-4167-8eab-1f763c111c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:05.506544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38060b-13c7-4167-8eab-1f763c111c1b None admin] PATH_INFO: `/v3/projects/3d52ac8525814206ade19cf3bc6b9271` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:05.507676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38060b-13c7-4167-8eab-1f763c111c1b None admin] RBAC: Authorizing `identity:get_project(project_id=3d52ac8525814206ade19cf3bc6b9271)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:05.509514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38060b-13c7-4167-8eab-1f763c111c1b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:05.516906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 217/436] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:30:05 2022] GET /identity/v3/projects/3d52ac8525814206ade19cf3bc6b9271 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:05.517785 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38060b-13c7-4167-8eab-1f763c111c1b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:05.552292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:05.555120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:05.575702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f8fd2-fd0a-41e4-8424-1c372111e164 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:05.624594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed6668-3041-4a40-a86b-f3b239ee538a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:05.628209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed6668-3041-4a40-a86b-f3b239ee538a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:05.628209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed6668-3041-4a40-a86b-f3b239ee538a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:05.628209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed6668-3041-4a40-a86b-f3b239ee538a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:05.628209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed6668-3041-4a40-a86b-f3b239ee538a None admin] RBAC: Adding query filter params (group.id=2d252c01f6974f079703843e664355e7, role.id=c70d64f54bff4d2ba00749c556be9c49, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:05.628209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed6668-3041-4a40-a86b-f3b239ee538a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:05.629697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed6668-3041-4a40-a86b-f3b239ee538a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:05.637273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 220/437] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Aug 31 11:30:05 2022] GET /identity/v3/role_assignments?group.id=2d252c01f6974f079703843e664355e7&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=c70d64f54bff4d2ba00749c556be9c49 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:05.638626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f130f1-dd3b-4ec9-b69d-7bbb3025e1b0 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:05.638909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f130f1-dd3b-4ec9-b69d-7bbb3025e1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:05.639186 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f130f1-dd3b-4ec9-b69d-7bbb3025e1b0 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:05.640756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 221/438] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:05.647022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408baadb-6f18-4e26-b981-011b1db654a4 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:05.647334 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408baadb-6f18-4e26-b981-011b1db654a4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:05.647641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408baadb-6f18-4e26-b981-011b1db654a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:05.658352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38060b-13c7-4167-8eab-1f763c111c1b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:05.669094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-408baadb-6f18-4e26-b981-011b1db654a4 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:05.670346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-408baadb-6f18-4e26-b981-011b1db654a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:05.710148 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ac48f-ddf7-470a-a77b-59107a7c212c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:05.711262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ac48f-ddf7-470a-a77b-59107a7c212c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:05.711788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ac48f-ddf7-470a-a77b-59107a7c212c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:05.712329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ac48f-ddf7-470a-a77b-59107a7c212c None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:05.714459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ac48f-ddf7-470a-a77b-59107a7c212c None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:05.716165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ac48f-ddf7-470a-a77b-59107a7c212c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:05.718490 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 218/439] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:30:05 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:05.724038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ac48f-ddf7-470a-a77b-59107a7c212c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:05.772912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 222/440] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:05.850611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:05.862312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ac48f-ddf7-470a-a77b-59107a7c212c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:05.899817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2bbb7-9ae2-47cc-bb43-3aa6a9c71591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:05.900418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2bbb7-9ae2-47cc-bb43-3aa6a9c71591 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:05.900823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2bbb7-9ae2-47cc-bb43-3aa6a9c71591 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:05.901611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2bbb7-9ae2-47cc-bb43-3aa6a9c71591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:05.902488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2bbb7-9ae2-47cc-bb43-3aa6a9c71591 None admin] RBAC: Adding query filter params (group.id=2d252c01f6974f079703843e664355e7, role.id=6f9c4ad710e84eaabc415a1dd1541055, scope.project.id=3d52ac8525814206ade19cf3bc6b9271) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:05.902793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2bbb7-9ae2-47cc-bb43-3aa6a9c71591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:05.904232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2bbb7-9ae2-47cc-bb43-3aa6a9c71591 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:05.910626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 219/441] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Aug 31 11:30:05 2022] GET /identity/v3/role_assignments?group.id=2d252c01f6974f079703843e664355e7&scope.project.id=3d52ac8525814206ade19cf3bc6b9271&role.id=6f9c4ad710e84eaabc415a1dd1541055 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:05.913053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf006-61c5-4f6c-9cd6-714c0ae14ce0 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:05.913344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf006-61c5-4f6c-9cd6-714c0ae14ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:05.913620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf006-61c5-4f6c-9cd6-714c0ae14ce0 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:05.914952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 223/442] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:05.921446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b0cc6-ea82-4f75-93ec-a6a91d87109a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:05.921748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b0cc6-ea82-4f75-93ec-a6a91d87109a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:05.921993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b0cc6-ea82-4f75-93ec-a6a91d87109a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:05.942648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3f4b0cc6-ea82-4f75-93ec-a6a91d87109a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:05.943885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3f4b0cc6-ea82-4f75-93ec-a6a91d87109a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:06.021965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 220/443] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:06.073943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbf006-61c5-4f6c-9cd6-714c0ae14ce0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:06.166905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:06.189594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbf006-61c5-4f6c-9cd6-714c0ae14ce0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:06.226476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb9c1a-832f-4858-9c7e-3d92cde4818e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:06.227109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9c1a-832f-4858-9c7e-3d92cde4818e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:06.227392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9c1a-832f-4858-9c7e-3d92cde4818e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:06.227671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9c1a-832f-4858-9c7e-3d92cde4818e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:06.238932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb9c1a-832f-4858-9c7e-3d92cde4818e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-74fb9c1a-832f-4858-9c7e-3d92cde4818e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:06.243826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:06.244768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:06.246345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 224/444] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:30:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:06.251299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b0cc6-ea82-4f75-93ec-a6a91d87109a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:06.294841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb9c1a-832f-4858-9c7e-3d92cde4818e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:06.350109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:06.371988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b0cc6-ea82-4f75-93ec-a6a91d87109a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:06.410019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331bcba-2f0d-4b70-b8e8-b75b30d49aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:06.410019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331bcba-2f0d-4b70-b8e8-b75b30d49aab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:06.410019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331bcba-2f0d-4b70-b8e8-b75b30d49aab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:06.410709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331bcba-2f0d-4b70-b8e8-b75b30d49aab None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:06.411067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331bcba-2f0d-4b70-b8e8-b75b30d49aab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:06.411367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331bcba-2f0d-4b70-b8e8-b75b30d49aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:06.412851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331bcba-2f0d-4b70-b8e8-b75b30d49aab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:06.420954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb9c1a-832f-4858-9c7e-3d92cde4818e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:06.421894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 221/445] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:06.428023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331bcba-2f0d-4b70-b8e8-b75b30d49aab None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:06.459235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187460e-abd8-4651-be05-67064cf3f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:06.459868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187460e-abd8-4651-be05-67064cf3f8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:06.460154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187460e-abd8-4651-be05-67064cf3f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:06.460433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187460e-abd8-4651-be05-67064cf3f8c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:06.470906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187460e-abd8-4651-be05-67064cf3f8c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-f187460e-abd8-4651-be05-67064cf3f8c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:06.475769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:06.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:06.481515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 225/446] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:06.483250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187460e-abd8-4651-be05-67064cf3f8c9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:06.486395 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:06.530932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331bcba-2f0d-4b70-b8e8-b75b30d49aab None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:06.564788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f609b9-395d-4c4b-ba39-5b0f938adf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:06.565385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f609b9-395d-4c4b-ba39-5b0f938adf46 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:06.565662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f609b9-395d-4c4b-ba39-5b0f938adf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:06.565933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f609b9-395d-4c4b-ba39-5b0f938adf46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:06.581821 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:06.583289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187460e-abd8-4651-be05-67064cf3f8c9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:06.617544 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84718f7-b205-4b70-9b52-5d681b825f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:06.618137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84718f7-b205-4b70-9b52-5d681b825f1a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:06.618425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84718f7-b205-4b70-9b52-5d681b825f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:06.618692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84718f7-b205-4b70-9b52-5d681b825f1a None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:06.619353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84718f7-b205-4b70-9b52-5d681b825f1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:06.619655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84718f7-b205-4b70-9b52-5d681b825f1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:06.621764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84718f7-b205-4b70-9b52-5d681b825f1a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:06.629532 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 226/447] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:06.634159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f609b9-395d-4c4b-ba39-5b0f938adf46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:06.634978 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84718f7-b205-4b70-9b52-5d681b825f1a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:06.635673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f609b9-395d-4c4b-ba39-5b0f938adf46 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:06.697418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:06.737150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84718f7-b205-4b70-9b52-5d681b825f1a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:06.737774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 222/448] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:06.743452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f609b9-395d-4c4b-ba39-5b0f938adf46 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:06.769656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3c1eb-6969-4a82-b5ea-8e6970f8b94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:06.770268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3c1eb-6969-4a82-b5ea-8e6970f8b94b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:06.770553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3c1eb-6969-4a82-b5ea-8e6970f8b94b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:06.770830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3c1eb-6969-4a82-b5ea-8e6970f8b94b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:06.789146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:06.845081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3c1eb-6969-4a82-b5ea-8e6970f8b94b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:06.846539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3c1eb-6969-4a82-b5ea-8e6970f8b94b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:06.847404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f609b9-395d-4c4b-ba39-5b0f938adf46 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:06.881308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3dda7-4195-41d9-8990-b3bff3ee2e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:06.881923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3dda7-4195-41d9-8990-b3bff3ee2e83 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:06.882197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3dda7-4195-41d9-8990-b3bff3ee2e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:06.882463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3dda7-4195-41d9-8990-b3bff3ee2e83 None admin] PATH_INFO: `/v3/projects/272ff39535e04ccb8b5d3e8facf8135c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:06.883581 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3dda7-4195-41d9-8990-b3bff3ee2e83 None admin] RBAC: Authorizing `identity:get_project(project_id=272ff39535e04ccb8b5d3e8facf8135c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:06.885029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3dda7-4195-41d9-8990-b3bff3ee2e83 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:06.886766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 223/449] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:30:06 2022] GET /identity/v3/projects/272ff39535e04ccb8b5d3e8facf8135c => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:06.892313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3dda7-4195-41d9-8990-b3bff3ee2e83 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:06.908634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:06.948672 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 227/450] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 11:30:06.954237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3c1eb-6969-4a82-b5ea-8e6970f8b94b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:06.993001 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3dda7-4195-41d9-8990-b3bff3ee2e83 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:07.007915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:07.027656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18d889-041e-4204-97ee-1482ce2b7767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:07.028272 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18d889-041e-4204-97ee-1482ce2b7767 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:07.028582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18d889-041e-4204-97ee-1482ce2b7767 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:07.028855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18d889-041e-4204-97ee-1482ce2b7767 None admin] PATH_INFO: `/v3/groups/2d252c01f6974f079703843e664355e7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:07.029959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18d889-041e-4204-97ee-1482ce2b7767 None admin] RBAC: Authorizing `identity:get_group(group_id=2d252c01f6974f079703843e664355e7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:07.031378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18d889-041e-4204-97ee-1482ce2b7767 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:07.032960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 224/451] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:30:06 2022] GET /identity/v3/groups/2d252c01f6974f079703843e664355e7 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:07.038570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18d889-041e-4204-97ee-1482ce2b7767 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:07.053026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3c1eb-6969-4a82-b5ea-8e6970f8b94b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:07.087782 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cb52e-dfad-4822-ab4f-168dd46ba2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:07.088405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cb52e-dfad-4822-ab4f-168dd46ba2de None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:07.088804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cb52e-dfad-4822-ab4f-168dd46ba2de None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:07.089087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cb52e-dfad-4822-ab4f-168dd46ba2de None admin] PATH_INFO: `/v3/projects/927949f45e9a418783820746376979d1` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:07.090232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64cb52e-dfad-4822-ab4f-168dd46ba2de None admin] RBAC: Authorizing `identity:get_project(project_id=927949f45e9a418783820746376979d1)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:07.091684 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64cb52e-dfad-4822-ab4f-168dd46ba2de None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:07.093330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 228/452] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:30:06 2022] GET /identity/v3/projects/927949f45e9a418783820746376979d1 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:07.098702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cb52e-dfad-4822-ab4f-168dd46ba2de None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:07.129128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:07.138609 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18d889-041e-4204-97ee-1482ce2b7767 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:07.173396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5e2f6-3e5b-4500-8d6e-9411ce36fa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:07.173998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5e2f6-3e5b-4500-8d6e-9411ce36fa21 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:07.174280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5e2f6-3e5b-4500-8d6e-9411ce36fa21 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:07.174960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5e2f6-3e5b-4500-8d6e-9411ce36fa21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:07.175684 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5e2f6-3e5b-4500-8d6e-9411ce36fa21 None admin] RBAC: Adding query filter params (group.id=2d252c01f6974f079703843e664355e7, role.id=6f9c4ad710e84eaabc415a1dd1541055, scope.project.id=272ff39535e04ccb8b5d3e8facf8135c) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:07.175997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5e2f6-3e5b-4500-8d6e-9411ce36fa21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:07.177454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5e2f6-3e5b-4500-8d6e-9411ce36fa21 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:07.184179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 225/453] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Aug 31 11:30:07 2022] GET /identity/v3/role_assignments?group.id=2d252c01f6974f079703843e664355e7&scope.project.id=272ff39535e04ccb8b5d3e8facf8135c&role.id=6f9c4ad710e84eaabc415a1dd1541055 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:07.200350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cb52e-dfad-4822-ab4f-168dd46ba2de None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:07.237561 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605635b-e9b1-41be-911f-12407f69bf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:07.238201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605635b-e9b1-41be-911f-12407f69bf17 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:07.238485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605635b-e9b1-41be-911f-12407f69bf17 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:07.238759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605635b-e9b1-41be-911f-12407f69bf17 None admin] PATH_INFO: `/v3/groups/94662cdd90d94c6ebec2ea4adacdc365` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:07.239924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605635b-e9b1-41be-911f-12407f69bf17 None admin] RBAC: Authorizing `identity:get_group(group_id=94662cdd90d94c6ebec2ea4adacdc365)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:07.241428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605635b-e9b1-41be-911f-12407f69bf17 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:07.243042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 229/454] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Aug 31 11:30:07 2022] GET /identity/v3/groups/94662cdd90d94c6ebec2ea4adacdc365 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:07.249391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5e2f6-3e5b-4500-8d6e-9411ce36fa21 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:07.299022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:07.360972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5e2f6-3e5b-4500-8d6e-9411ce36fa21 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:07.394631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43551c-52bd-4bbc-a71e-13bcb86dc3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:07.395234 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43551c-52bd-4bbc-a71e-13bcb86dc3fe None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:07.395512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43551c-52bd-4bbc-a71e-13bcb86dc3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:07.395780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43551c-52bd-4bbc-a71e-13bcb86dc3fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:07.396517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43551c-52bd-4bbc-a71e-13bcb86dc3fe None admin] RBAC: Adding query filter params (group.id=94662cdd90d94c6ebec2ea4adacdc365, role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=927949f45e9a418783820746376979d1) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:07.396824 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43551c-52bd-4bbc-a71e-13bcb86dc3fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:07.398232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43551c-52bd-4bbc-a71e-13bcb86dc3fe None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:07.406583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 226/455] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Aug 31 11:30:07 2022] GET /identity/v3/role_assignments?group.id=94662cdd90d94c6ebec2ea4adacdc365&scope.project.id=927949f45e9a418783820746376979d1&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:10.056187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b170a9c-d24b-47dd-ad1a-db032f25c15c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:10.056722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b170a9c-d24b-47dd-ad1a-db032f25c15c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:10.056722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b170a9c-d24b-47dd-ad1a-db032f25c15c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:10.058539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 230/456] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:10.063962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502533ac-9d4a-4db8-86e1-3eade06c9c11 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:10.064393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502533ac-9d4a-4db8-86e1-3eade06c9c11 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:10.064544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502533ac-9d4a-4db8-86e1-3eade06c9c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:10.097555 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-502533ac-9d4a-4db8-86e1-3eade06c9c11 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:10.098740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-502533ac-9d4a-4db8-86e1-3eade06c9c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:10.258699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 227/457] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:30:10.332616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c8487-6d28-4322-89f3-b5a018fb1024 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:10.332907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c8487-6d28-4322-89f3-b5a018fb1024 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:10.333193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c8487-6d28-4322-89f3-b5a018fb1024 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:10.336872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725b602-bdc5-486e-95bd-4ca31690b0f5 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:10.337160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725b602-bdc5-486e-95bd-4ca31690b0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:10.337430 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725b602-bdc5-486e-95bd-4ca31690b0f5 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:10.339190 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 228/459] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:30:10.339317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 231/459] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:10.343822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8cfe2-f6bd-4b7b-a5c8-41cc74462e49 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:10.343947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9099727-993a-40e7-821c-599f333e214e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:10.344104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8cfe2-f6bd-4b7b-a5c8-41cc74462e49 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:10.344208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9099727-993a-40e7-821c-599f333e214e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:10.344363 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8cfe2-f6bd-4b7b-a5c8-41cc74462e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:10.344514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9099727-993a-40e7-821c-599f333e214e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:10.363190 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e7d8cfe2-f6bd-4b7b-a5c8-41cc74462e49 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:10.364413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e7d8cfe2-f6bd-4b7b-a5c8-41cc74462e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:10.375710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c9099727-993a-40e7-821c-599f333e214e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:10.376935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c9099727-993a-40e7-821c-599f333e214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:10.428851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:10.469947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 229/460] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:10.546084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 232/461] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:10.624507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8cfe2-f6bd-4b7b-a5c8-41cc74462e49 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:10.732609 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb9eaf-05da-46cc-885d-77f34a16cd8f None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:10.733248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb9eaf-05da-46cc-885d-77f34a16cd8f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:10.733724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb9eaf-05da-46cc-885d-77f34a16cd8f None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:10.735514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 233/462] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:10.740647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b453d1-5083-4455-850a-dacc1d113984 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:10.741155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b453d1-5083-4455-850a-dacc1d113984 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:10.741708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b453d1-5083-4455-850a-dacc1d113984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:10.776982 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8cfe2-f6bd-4b7b-a5c8-41cc74462e49 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:10.781688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-80b453d1-5083-4455-850a-dacc1d113984 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:10.783090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-80b453d1-5083-4455-850a-dacc1d113984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:10.817301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b8893-d19b-43e9-be4e-bcb86e434f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:10.818465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b8893-d19b-43e9-be4e-bcb86e434f0c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:10.819214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b8893-d19b-43e9-be4e-bcb86e434f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:10.819933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b8893-d19b-43e9-be4e-bcb86e434f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:10.832510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b8893-d19b-43e9-be4e-bcb86e434f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:10.833121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:10.835091 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b8893-d19b-43e9-be4e-bcb86e434f0c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-fb9b8893-d19b-43e9-be4e-bcb86e434f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:10.855844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:10.856838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:10.857696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:10.857696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:10.857696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:10.858576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 234/463] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:30:10.864514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b453d1-5083-4455-850a-dacc1d113984 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:10.865431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 230/464] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:10.881423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b8893-d19b-43e9-be4e-bcb86e434f0c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:10.916134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:10.994040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b453d1-5083-4455-850a-dacc1d113984 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:11.019432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b8893-d19b-43e9-be4e-bcb86e434f0c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:11.030835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e65bb-3cfd-4165-beed-b2514d28223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:11.031479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e65bb-3cfd-4165-beed-b2514d28223b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.031761 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e65bb-3cfd-4165-beed-b2514d28223b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.032200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e65bb-3cfd-4165-beed-b2514d28223b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.044880 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e65bb-3cfd-4165-beed-b2514d28223b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:11.047230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e65bb-3cfd-4165-beed-b2514d28223b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:11.063181 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf37e9-5fb2-495f-9497-a1f2873ae225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:11.064152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf37e9-5fb2-495f-9497-a1f2873ae225 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.065195 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf37e9-5fb2-495f-9497-a1f2873ae225 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.065697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf37e9-5fb2-495f-9497-a1f2873ae225 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-0d7e65bb-3cfd-4165-beed-b2514d28223b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:11.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:11.067214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:11.068076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:11.068076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:11.068076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:11.068076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 235/465] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:11.068076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e65bb-3cfd-4165-beed-b2514d28223b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:11.069766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf37e9-5fb2-495f-9497-a1f2873ae225 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:11.070285 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf37e9-5fb2-495f-9497-a1f2873ae225 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:11.073561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf37e9-5fb2-495f-9497-a1f2873ae225 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:11.091852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:11.096229 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 231/466] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:11.098065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf37e9-5fb2-495f-9497-a1f2873ae225 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:11.216206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e65bb-3cfd-4165-beed-b2514d28223b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:11.232799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:11.255605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf37e9-5fb2-495f-9497-a1f2873ae225 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:11.260954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcc1b1-3a03-4668-99ea-01d390d212e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:11.261594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcc1b1-3a03-4668-99ea-01d390d212e1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.261874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcc1b1-3a03-4668-99ea-01d390d212e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.262152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcc1b1-3a03-4668-99ea-01d390d212e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.276522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcc1b1-3a03-4668-99ea-01d390d212e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:11.278044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcc1b1-3a03-4668-99ea-01d390d212e1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-2bfcc1b1-3a03-4668-99ea-01d390d212e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:11.294303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:11.295251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:11.296107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:11.296107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:11.296107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:11.296107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 236/467] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:11.296323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcc1b1-3a03-4668-99ea-01d390d212e1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:11.302568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96da7f2-2861-4dce-87d4-a478caf3e1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:11.303656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96da7f2-2861-4dce-87d4-a478caf3e1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.304233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96da7f2-2861-4dce-87d4-a478caf3e1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.304846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96da7f2-2861-4dce-87d4-a478caf3e1c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.305832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96da7f2-2861-4dce-87d4-a478caf3e1c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:11.306423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96da7f2-2861-4dce-87d4-a478caf3e1c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:11.308078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96da7f2-2861-4dce-87d4-a478caf3e1c2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:11.330418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 232/468] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:11.330418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96da7f2-2861-4dce-87d4-a478caf3e1c2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:11.371811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:11.414013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcc1b1-3a03-4668-99ea-01d390d212e1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:11.452500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96da7f2-2861-4dce-87d4-a478caf3e1c2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:11.453625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75796295-6027-41b6-b8a1-18dd4719158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:11.454229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75796295-6027-41b6-b8a1-18dd4719158b None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.454507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75796295-6027-41b6-b8a1-18dd4719158b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.455523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75796295-6027-41b6-b8a1-18dd4719158b None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.455523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75796295-6027-41b6-b8a1-18dd4719158b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:11.456895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75796295-6027-41b6-b8a1-18dd4719158b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:11.485619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 237/469] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:30:11 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:30:11.486224 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75796295-6027-41b6-b8a1-18dd4719158b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:11.492855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a58ea-39fa-4d80-bde7-14cb30927ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:11.493537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a58ea-39fa-4d80-bde7-14cb30927ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.493819 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a58ea-39fa-4d80-bde7-14cb30927ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.494098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a58ea-39fa-4d80-bde7-14cb30927ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.505874 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a58ea-39fa-4d80-bde7-14cb30927ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:11.507419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a58ea-39fa-4d80-bde7-14cb30927ce4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:11.509514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-371a58ea-39fa-4d80-bde7-14cb30927ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:11.528803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:11.529845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:11.530760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:11.530760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:11.530760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:11.532312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 233/470] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:11.533396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a58ea-39fa-4d80-bde7-14cb30927ce4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:11.630827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75796295-6027-41b6-b8a1-18dd4719158b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:11.659976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:11.668007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a58ea-39fa-4d80-bde7-14cb30927ce4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:11.684910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a09824-b2bc-4baf-a912-4b80701509e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:11.685883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a09824-b2bc-4baf-a912-4b80701509e9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.686356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a09824-b2bc-4baf-a912-4b80701509e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.686814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a09824-b2bc-4baf-a912-4b80701509e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.687894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a09824-b2bc-4baf-a912-4b80701509e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:11.688391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a09824-b2bc-4baf-a912-4b80701509e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:11.690314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a09824-b2bc-4baf-a912-4b80701509e9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:11.711827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 238/471] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:11.714672 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af26bf0-3cd0-4781-bb92-544df3142ec3 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.714958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af26bf0-3cd0-4781-bb92-544df3142ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.715224 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af26bf0-3cd0-4781-bb92-544df3142ec3 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.718246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 239/472] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:11.718246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af26bf0-3cd0-4781-bb92-544df3142ec3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:11.721647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145370c-a069-448b-b959-3e64d8e1f752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:11.721647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145370c-a069-448b-b959-3e64d8e1f752 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.721647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145370c-a069-448b-b959-3e64d8e1f752 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.721647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145370c-a069-448b-b959-3e64d8e1f752 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.721996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145370c-a069-448b-b959-3e64d8e1f752 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:11.723588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145370c-a069-448b-b959-3e64d8e1f752 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:11.755911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 234/473] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:30:11 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:30:11.756443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145370c-a069-448b-b959-3e64d8e1f752 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:11.822109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:11.897719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145370c-a069-448b-b959-3e64d8e1f752 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:11.910335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af26bf0-3cd0-4781-bb92-544df3142ec3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:11.939326 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d69a1-2dda-4413-bf03-f678371ef45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:11.940018 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d69a1-2dda-4413-bf03-f678371ef45f None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.940210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d69a1-2dda-4413-bf03-f678371ef45f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.940917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d69a1-2dda-4413-bf03-f678371ef45f None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.941828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d69a1-2dda-4413-bf03-f678371ef45f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:11.943524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d69a1-2dda-4413-bf03-f678371ef45f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:11.971334 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f0013-6b46-41db-ae16-b0f422a61088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:11.971960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f0013-6b46-41db-ae16-b0f422a61088 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.972249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f0013-6b46-41db-ae16-b0f422a61088 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.973448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f0013-6b46-41db-ae16-b0f422a61088 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.973448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f0013-6b46-41db-ae16-b0f422a61088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:11.973620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f0013-6b46-41db-ae16-b0f422a61088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:11.975198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f0013-6b46-41db-ae16-b0f422a61088 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:11.975311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 235/474] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:30:11 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:30:11.976459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c36a80-73ae-4766-8e29-1097b69b237a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:11.976780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c36a80-73ae-4766-8e29-1097b69b237a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:11.977801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c36a80-73ae-4766-8e29-1097b69b237a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:11.999455 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 240/475] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:12.006026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-34c36a80-73ae-4766-8e29-1097b69b237a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:12.006197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f0013-6b46-41db-ae16-b0f422a61088 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:12.007264 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-34c36a80-73ae-4766-8e29-1097b69b237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:12.070400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:12.108545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 236/476] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:12.180443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f0013-6b46-41db-ae16-b0f422a61088 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:12.223862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c377d-522f-4a14-9afb-275cb055e52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:12.224647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c377d-522f-4a14-9afb-275cb055e52d None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:12.225029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c377d-522f-4a14-9afb-275cb055e52d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:12.225427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c377d-522f-4a14-9afb-275cb055e52d None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:12.226184 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c377d-522f-4a14-9afb-275cb055e52d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:12.227746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c377d-522f-4a14-9afb-275cb055e52d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:12.255115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 241/477] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:30:12 2022] POST /identity/v3/users => generated 249 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:30:12.403771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c36a80-73ae-4766-8e29-1097b69b237a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:12.460753 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:12.528732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c36a80-73ae-4766-8e29-1097b69b237a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:12.569687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183e1c6-8ace-4228-a71e-41787332acec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:12.570304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183e1c6-8ace-4228-a71e-41787332acec None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:12.570584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183e1c6-8ace-4228-a71e-41787332acec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:12.570857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183e1c6-8ace-4228-a71e-41787332acec None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:12.571541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183e1c6-8ace-4228-a71e-41787332acec None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:12.573488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183e1c6-8ace-4228-a71e-41787332acec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:12.583002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.sql.core [None req-e183e1c6-8ace-4228-a71e-41787332acec None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 11:30:12.583002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 11:30:12.583002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [parameters: {'id': '32be3fda5d5746899ad7d2a006be8b3c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 11:30:12.583002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81620) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e183e1c6-8ace-4228-a71e-41787332acec None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Aug 31 11:30:12.600099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 11:30:12.601040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application yield resource Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise self.value Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application session.commit() Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 11:30:12.601973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.flush() Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application compat.raise_( Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 11:30:12.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 11:30:12.603688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 11:30:12.607980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [parameters: {'id': '32be3fda5d5746899ad7d2a006be8b3c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:12.609910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 11:30:12.611814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:12.613745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 237/478] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Aug 31 11:30:12 2022] POST /identity/v3/roles => generated 144 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:12.613877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c377d-522f-4a14-9afb-275cb055e52d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:12.851051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:12.856693 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c377d-522f-4a14-9afb-275cb055e52d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:12.950632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06759b6-011c-4202-9c0b-bb3956d5fd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:12.951192 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06759b6-011c-4202-9c0b-bb3956d5fd66 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:12.951472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06759b6-011c-4202-9c0b-bb3956d5fd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:12.951740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06759b6-011c-4202-9c0b-bb3956d5fd66 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:12.971349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06759b6-011c-4202-9c0b-bb3956d5fd66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c06759b6-011c-4202-9c0b-bb3956d5fd66 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:12.976195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:12.977181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:12.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 242/479] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:30:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:12.985576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183e1c6-8ace-4228-a71e-41787332acec None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:13.137139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:13.143451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183e1c6-8ace-4228-a71e-41787332acec None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:13.184025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a9a34-ef85-42ec-a4e3-bbea44835332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:13.184697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9a34-ef85-42ec-a4e3-bbea44835332 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:13.184978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9a34-ef85-42ec-a4e3-bbea44835332 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:13.185248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9a34-ef85-42ec-a4e3-bbea44835332 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:13.185904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a9a34-ef85-42ec-a4e3-bbea44835332 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:13.186207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a9a34-ef85-42ec-a4e3-bbea44835332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:13.187621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a9a34-ef85-42ec-a4e3-bbea44835332 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:13.196417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 238/480] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:13.852516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08db5f5-34b0-411f-8109-48ca2d156db6 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:13.852516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08db5f5-34b0-411f-8109-48ca2d156db6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:13.852516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08db5f5-34b0-411f-8109-48ca2d156db6 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:13.852516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 243/481] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:13.866424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5b931-c1d4-441c-8aa7-2bac22d15f45 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:13.872515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5b931-c1d4-441c-8aa7-2bac22d15f45 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:13.872515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5b931-c1d4-441c-8aa7-2bac22d15f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:13.924522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d1f5b931-c1d4-441c-8aa7-2bac22d15f45 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:13.924522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d1f5b931-c1d4-441c-8aa7-2bac22d15f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:14.044076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 239/482] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:14.505780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460c8c6-7156-452e-95ac-3c54d45b3d6e None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:14.506170 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460c8c6-7156-452e-95ac-3c54d45b3d6e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:14.506343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460c8c6-7156-452e-95ac-3c54d45b3d6e None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:14.508509 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 244/483] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:14.514694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f336da-99bd-49c3-b0b8-8f2aa849886e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:14.520676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f336da-99bd-49c3-b0b8-8f2aa849886e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:14.523608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f336da-99bd-49c3-b0b8-8f2aa849886e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:14.544803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460c8c6-7156-452e-95ac-3c54d45b3d6e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:14.549701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b6f336da-99bd-49c3-b0b8-8f2aa849886e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:14.553343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b6f336da-99bd-49c3-b0b8-8f2aa849886e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:14.760862 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 240/484] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:14.787271 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125c45e-97e3-4419-a9ea-1b83b6f9fc07 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:14.787562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125c45e-97e3-4419-a9ea-1b83b6f9fc07 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:14.792516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125c45e-97e3-4419-a9ea-1b83b6f9fc07 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:14.792516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 241/485] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:14.795186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02913659-18ea-4136-883a-07d11545cfac None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:14.798888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02913659-18ea-4136-883a-07d11545cfac None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:14.798888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02913659-18ea-4136-883a-07d11545cfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:14.860516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-02913659-18ea-4136-883a-07d11545cfac None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:14.860516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-02913659-18ea-4136-883a-07d11545cfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:14.886012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460c8c6-7156-452e-95ac-3c54d45b3d6e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:14.895094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:14.952594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315c844-2338-413d-b6ac-571e22a4ae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:14.953228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315c844-2338-413d-b6ac-571e22a4ae8c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:14.953512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315c844-2338-413d-b6ac-571e22a4ae8c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:14.953794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315c844-2338-413d-b6ac-571e22a4ae8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:14.980732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7315c844-2338-413d-b6ac-571e22a4ae8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-7315c844-2338-413d-b6ac-571e22a4ae8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:14.986858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:14.987721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:14.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:14.999887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 245/486] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:15.003142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a29f4-3d33-48a0-9848-d8d1e6dc403d None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:15.003438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a29f4-3d33-48a0-9848-d8d1e6dc403d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:15.003717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a29f4-3d33-48a0-9848-d8d1e6dc403d None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:15.007545 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 246/487] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:15.008268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a29f4-3d33-48a0-9848-d8d1e6dc403d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:15.011659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 242/488] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:15.016720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3a734-6a9a-4575-bbef-2f81b1f9c4de None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:15.016720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3a734-6a9a-4575-bbef-2f81b1f9c4de None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:15.016720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3a734-6a9a-4575-bbef-2f81b1f9c4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:15.071163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:15.074911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0ab3a734-6a9a-4575-bbef-2f81b1f9c4de None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:15.076230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0ab3a734-6a9a-4575-bbef-2f81b1f9c4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:15.234967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 243/489] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:15.235943 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3a734-6a9a-4575-bbef-2f81b1f9c4de None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:15.252511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a29f4-3d33-48a0-9848-d8d1e6dc403d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:15.318496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61315d1a-37d9-4218-8280-ca2d8aa01014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:15.319116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61315d1a-37d9-4218-8280-ca2d8aa01014 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:15.319400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61315d1a-37d9-4218-8280-ca2d8aa01014 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:15.319674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61315d1a-37d9-4218-8280-ca2d8aa01014 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:15.320664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61315d1a-37d9-4218-8280-ca2d8aa01014 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:15.320975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61315d1a-37d9-4218-8280-ca2d8aa01014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:15.322615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61315d1a-37d9-4218-8280-ca2d8aa01014 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:15.336626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 247/490] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:15.337830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61315d1a-37d9-4218-8280-ca2d8aa01014 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:15.366770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:15.393343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3a734-6a9a-4575-bbef-2f81b1f9c4de None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:15.431631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4201e3e-6c8e-4567-be98-a34badbc4e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:15.434275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4201e3e-6c8e-4567-be98-a34badbc4e88 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:15.435616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4201e3e-6c8e-4567-be98-a34badbc4e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:15.436978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4201e3e-6c8e-4567-be98-a34badbc4e88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:15.451916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4201e3e-6c8e-4567-be98-a34badbc4e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:15.459225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e4201e3e-6c8e-4567-be98-a34badbc4e88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:15.459816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:15.461084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:15.461891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:15.464740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 244/491] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:15.465765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4201e3e-6c8e-4567-be98-a34badbc4e88 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:15.502744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61315d1a-37d9-4218-8280-ca2d8aa01014 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:15.555312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d84d51-e755-40db-b1da-878cf2fb9803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:15.556070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84d51-e755-40db-b1da-878cf2fb9803 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:15.556388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84d51-e755-40db-b1da-878cf2fb9803 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:15.556704 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84d51-e755-40db-b1da-878cf2fb9803 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:15.568871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d84d51-e755-40db-b1da-878cf2fb9803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-91d84d51-e755-40db-b1da-878cf2fb9803 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:15.575038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:15.575914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 248/492] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:15.581514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d84d51-e755-40db-b1da-878cf2fb9803 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:15.696433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4201e3e-6c8e-4567-be98-a34badbc4e88 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:15.724479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d84d51-e755-40db-b1da-878cf2fb9803 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:15.736745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808a421-c894-44b0-85d5-417471409cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:15.737375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808a421-c894-44b0-85d5-417471409cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:15.737650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808a421-c894-44b0-85d5-417471409cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:15.737926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808a421-c894-44b0-85d5-417471409cc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:15.749589 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808a421-c894-44b0-85d5-417471409cc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:15.751065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808a421-c894-44b0-85d5-417471409cc5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:15.753432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:15.763735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db3b26-79b5-4dff-a06a-f66e5b52f68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:15.764340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db3b26-79b5-4dff-a06a-f66e5b52f68a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:15.764686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db3b26-79b5-4dff-a06a-f66e5b52f68a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:15.764956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db3b26-79b5-4dff-a06a-f66e5b52f68a None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:15.765619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db3b26-79b5-4dff-a06a-f66e5b52f68a None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:15.765929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db3b26-79b5-4dff-a06a-f66e5b52f68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:15.767280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db3b26-79b5-4dff-a06a-f66e5b52f68a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a808a421-c894-44b0-85d5-417471409cc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:15.769870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:15.770726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:15.771551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:15.771551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:15.771551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:15.771551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 245/493] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:15.771769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808a421-c894-44b0-85d5-417471409cc5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:15.777360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:15.784114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 249/494] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:15.786227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db3b26-79b5-4dff-a06a-f66e5b52f68a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:15.892110 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808a421-c894-44b0-85d5-417471409cc5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:15.907387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db3b26-79b5-4dff-a06a-f66e5b52f68a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:15.930945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9bc3f-4486-4bdd-aafb-3db20c6b70c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:15.931578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9bc3f-4486-4bdd-aafb-3db20c6b70c8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:15.931852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9bc3f-4486-4bdd-aafb-3db20c6b70c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:15.932737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9bc3f-4486-4bdd-aafb-3db20c6b70c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:15.944192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9bc3f-4486-4bdd-aafb-3db20c6b70c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:15.945554 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56624f9-8a1c-446b-a6e3-ae3cc2865db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:15.946154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56624f9-8a1c-446b-a6e3-ae3cc2865db9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:15.946434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56624f9-8a1c-446b-a6e3-ae3cc2865db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:15.946709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56624f9-8a1c-446b-a6e3-ae3cc2865db9 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:15.947413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56624f9-8a1c-446b-a6e3-ae3cc2865db9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:15.947765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56624f9-8a1c-446b-a6e3-ae3cc2865db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:15.949707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56624f9-8a1c-446b-a6e3-ae3cc2865db9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d5b9bc3f-4486-4bdd-aafb-3db20c6b70c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:15.950731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:15.951569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:15.952367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 246/495] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:15.953070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440f2a-3eaa-4dd6-863b-4b81538cbc47 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:15.953070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440f2a-3eaa-4dd6-863b-4b81538cbc47 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:15.953070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440f2a-3eaa-4dd6-863b-4b81538cbc47 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:15.954374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 247/496] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:15.958904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c440f2a-3eaa-4dd6-863b-4b81538cbc47 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:15.964026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 250/497] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:15.964026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56624f9-8a1c-446b-a6e3-ae3cc2865db9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:16.023563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:16.038208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:16.076433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c440f2a-3eaa-4dd6-863b-4b81538cbc47 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:16.082245 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56624f9-8a1c-446b-a6e3-ae3cc2865db9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:16.112771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ebabf-3ae6-4f7c-b40e-9e82a2af9ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:16.113502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ebabf-3ae6-4f7c-b40e-9e82a2af9ebc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.113872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ebabf-3ae6-4f7c-b40e-9e82a2af9ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.114249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ebabf-3ae6-4f7c-b40e-9e82a2af9ebc None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.114975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ebabf-3ae6-4f7c-b40e-9e82a2af9ebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:16.115346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ebabf-3ae6-4f7c-b40e-9e82a2af9ebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:16.116259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392abfb-4f7b-48a2-927c-91f26765b401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:16.116870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ebabf-3ae6-4f7c-b40e-9e82a2af9ebc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:16.116965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392abfb-4f7b-48a2-927c-91f26765b401 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.117187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392abfb-4f7b-48a2-927c-91f26765b401 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.117471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392abfb-4f7b-48a2-927c-91f26765b401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.133193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 248/498] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:16.133193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ebabf-3ae6-4f7c-b40e-9e82a2af9ebc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:16.133450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392abfb-4f7b-48a2-927c-91f26765b401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:16.133862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392abfb-4f7b-48a2-927c-91f26765b401 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-5392abfb-4f7b-48a2-927c-91f26765b401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:16.152267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:16.158006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:16.158926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:16.158926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:16.158926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:16.158926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 251/499] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:16.158926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005af1c-bef4-4216-97ef-58dde76e5349 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.158926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005af1c-bef4-4216-97ef-58dde76e5349 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.158926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005af1c-bef4-4216-97ef-58dde76e5349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.173295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6005af1c-bef4-4216-97ef-58dde76e5349 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:16.174498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6005af1c-bef4-4216-97ef-58dde76e5349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:16.244777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 252/500] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:16.244976 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ebabf-3ae6-4f7c-b40e-9e82a2af9ebc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:16.245072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005af1c-bef4-4216-97ef-58dde76e5349 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:16.259840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:16.287273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa89f5b-de9e-4d4f-b6cf-fb166da60d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:16.288507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa89f5b-de9e-4d4f-b6cf-fb166da60d96 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.288937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa89f5b-de9e-4d4f-b6cf-fb166da60d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.289352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa89f5b-de9e-4d4f-b6cf-fb166da60d96 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.290534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa89f5b-de9e-4d4f-b6cf-fb166da60d96 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:16.290989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa89f5b-de9e-4d4f-b6cf-fb166da60d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:16.293822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa89f5b-de9e-4d4f-b6cf-fb166da60d96 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:16.294504 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:16.304260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 249/501] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:16.305098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa89f5b-de9e-4d4f-b6cf-fb166da60d96 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:16.362295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005af1c-bef4-4216-97ef-58dde76e5349 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:16.398863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42920dc0-b8a6-4cab-a39b-f45704ee09a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:16.399498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42920dc0-b8a6-4cab-a39b-f45704ee09a0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.400181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42920dc0-b8a6-4cab-a39b-f45704ee09a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.400469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42920dc0-b8a6-4cab-a39b-f45704ee09a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.413069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa89f5b-de9e-4d4f-b6cf-fb166da60d96 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:16.414123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42920dc0-b8a6-4cab-a39b-f45704ee09a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:16.417340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42920dc0-b8a6-4cab-a39b-f45704ee09a0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-42920dc0-b8a6-4cab-a39b-f45704ee09a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:16.435740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:16.436866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:16.437695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:16.437695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:16.437695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:16.437695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 253/502] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:16.438501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42920dc0-b8a6-4cab-a39b-f45704ee09a0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:16.449855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5348-d23f-4abd-9661-5826d14ca404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:16.450477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5348-d23f-4abd-9661-5826d14ca404 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.450761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5348-d23f-4abd-9661-5826d14ca404 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.451037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5348-d23f-4abd-9661-5826d14ca404 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.459011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e5348-d23f-4abd-9661-5826d14ca404 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:16.460466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e5348-d23f-4abd-9661-5826d14ca404 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-544e5348-d23f-4abd-9661-5826d14ca404 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:16.474636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:16.475535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:16.476427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:16.476427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:16.476427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:16.476427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:16.476427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:16.476427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:16.476427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:16.476427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:16.476427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:16.476427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:16.476427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 250/503] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Aug 31 11:30:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:16.477055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5348-d23f-4abd-9661-5826d14ca404 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:16.515363 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:16.551050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:16.559190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42920dc0-b8a6-4cab-a39b-f45704ee09a0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:16.582130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5348-d23f-4abd-9661-5826d14ca404 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:16.593856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf779e3e-ef8b-4e43-b26e-38d6fc6a476e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:16.594463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf779e3e-ef8b-4e43-b26e-38d6fc6a476e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.594742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf779e3e-ef8b-4e43-b26e-38d6fc6a476e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.595015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf779e3e-ef8b-4e43-b26e-38d6fc6a476e None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.595655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf779e3e-ef8b-4e43-b26e-38d6fc6a476e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:16.595956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf779e3e-ef8b-4e43-b26e-38d6fc6a476e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:16.597353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf779e3e-ef8b-4e43-b26e-38d6fc6a476e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:16.608640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 254/504] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:16.609428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf779e3e-ef8b-4e43-b26e-38d6fc6a476e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:16.618769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c89d6-a853-492f-9178-3aef1625640e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:16.619391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c89d6-a853-492f-9178-3aef1625640e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.620223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c89d6-a853-492f-9178-3aef1625640e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.620477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c89d6-a853-492f-9178-3aef1625640e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.631152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c89d6-a853-492f-9178-3aef1625640e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:16.632642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c89d6-a853-492f-9178-3aef1625640e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e70c89d6-a853-492f-9178-3aef1625640e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:16.647211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:16.648093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:16.648969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:16.648969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:16.648969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:16.648969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 251/505] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:16.649180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c89d6-a853-492f-9178-3aef1625640e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:16.710579 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf779e3e-ef8b-4e43-b26e-38d6fc6a476e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:16.744072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c32100-4fe3-4f79-b8dd-90a367981acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:16.744759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32100-4fe3-4f79-b8dd-90a367981acb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.745040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32100-4fe3-4f79-b8dd-90a367981acb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.745314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32100-4fe3-4f79-b8dd-90a367981acb None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.745953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c32100-4fe3-4f79-b8dd-90a367981acb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:16.746261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c32100-4fe3-4f79-b8dd-90a367981acb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:16.747560 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:16.748008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c32100-4fe3-4f79-b8dd-90a367981acb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:16.759929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 255/506] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:16.760109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c89d6-a853-492f-9178-3aef1625640e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:16.761640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c32100-4fe3-4f79-b8dd-90a367981acb None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:16.780937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:16.795562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a022088-2c80-455b-97f4-486dc5c46e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:16.796169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a022088-2c80-455b-97f4-486dc5c46e4b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.796450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a022088-2c80-455b-97f4-486dc5c46e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.796796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a022088-2c80-455b-97f4-486dc5c46e4b None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.797524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a022088-2c80-455b-97f4-486dc5c46e4b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:16.797837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a022088-2c80-455b-97f4-486dc5c46e4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:16.799313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a022088-2c80-455b-97f4-486dc5c46e4b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:16.837019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 252/507] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Aug 31 11:30:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:16.837840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a022088-2c80-455b-97f4-486dc5c46e4b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:16.865284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c32100-4fe3-4f79-b8dd-90a367981acb None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:16.900086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c49c0-c552-40c3-b895-37e75c6b9631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:16.900744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c49c0-c552-40c3-b895-37e75c6b9631 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.901362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c49c0-c552-40c3-b895-37e75c6b9631 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.901750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c49c0-c552-40c3-b895-37e75c6b9631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.912002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c49c0-c552-40c3-b895-37e75c6b9631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:16.913497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c49c0-c552-40c3-b895-37e75c6b9631 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c87c49c0-c552-40c3-b895-37e75c6b9631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:16.928109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:16.929076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:16.929943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:16.929943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:16.929943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:16.929943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 256/508] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:16.930163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c49c0-c552-40c3-b895-37e75c6b9631 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:16.945141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a022088-2c80-455b-97f4-486dc5c46e4b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:16.979947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c446e-8cae-4b5b-a2a1-13282e56c52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:16.980592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c446e-8cae-4b5b-a2a1-13282e56c52b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:16.980875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c446e-8cae-4b5b-a2a1-13282e56c52b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:16.981151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c446e-8cae-4b5b-a2a1-13282e56c52b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:16.988330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c446e-8cae-4b5b-a2a1-13282e56c52b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:16.989807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c446e-8cae-4b5b-a2a1-13282e56c52b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.000902 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-4e9c446e-8cae-4b5b-a2a1-13282e56c52b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:17.002096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:17.003062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 253/509] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:17.003990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c446e-8cae-4b5b-a2a1-13282e56c52b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:17.013776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:17.038911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c49c0-c552-40c3-b895-37e75c6b9631 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:17.073480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e9c56-89a0-4305-8a19-b807153fc4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:17.074085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e9c56-89a0-4305-8a19-b807153fc4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:17.074362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e9c56-89a0-4305-8a19-b807153fc4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:17.074634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e9c56-89a0-4305-8a19-b807153fc4e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:17.075277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e9c56-89a0-4305-8a19-b807153fc4e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:17.075582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e9c56-89a0-4305-8a19-b807153fc4e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:17.076998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e9c56-89a0-4305-8a19-b807153fc4e3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.089116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 257/510] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:17.089918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e9c56-89a0-4305-8a19-b807153fc4e3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:17.114813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c446e-8cae-4b5b-a2a1-13282e56c52b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:17.152247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d67c7-4241-4794-a055-971fcb508dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:17.152902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d67c7-4241-4794-a055-971fcb508dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:17.153180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d67c7-4241-4794-a055-971fcb508dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:17.153453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d67c7-4241-4794-a055-971fcb508dc9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:17.160689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d67c7-4241-4794-a055-971fcb508dc9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:17.162151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d67c7-4241-4794-a055-971fcb508dc9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-593d67c7-4241-4794-a055-971fcb508dc9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:17.173483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:17.174395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 254/511] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:17.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d67c7-4241-4794-a055-971fcb508dc9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:17.193723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e9c56-89a0-4305-8a19-b807153fc4e3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:17.221109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:17.229401 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324098f-9f1d-4b99-97a3-04e4d0937f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:17.230009 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324098f-9f1d-4b99-97a3-04e4d0937f31 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:17.230291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324098f-9f1d-4b99-97a3-04e4d0937f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:17.230564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324098f-9f1d-4b99-97a3-04e4d0937f31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:17.240801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:17.275587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d67c7-4241-4794-a055-971fcb508dc9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:17.299064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324098f-9f1d-4b99-97a3-04e4d0937f31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:17.300552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324098f-9f1d-4b99-97a3-04e4d0937f31 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.309619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f75c7-e907-4780-96c1-cefc5138a2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:17.310225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f75c7-e907-4780-96c1-cefc5138a2bd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:17.310515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f75c7-e907-4780-96c1-cefc5138a2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:17.310784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f75c7-e907-4780-96c1-cefc5138a2bd None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:17.311437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f75c7-e907-4780-96c1-cefc5138a2bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:17.311742 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f75c7-e907-4780-96c1-cefc5138a2bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:17.313170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f75c7-e907-4780-96c1-cefc5138a2bd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.325461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 255/512] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:17.326253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f75c7-e907-4780-96c1-cefc5138a2bd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:17.404159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 258/513] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:17.405159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324098f-9f1d-4b99-97a3-04e4d0937f31 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:17.432533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f75c7-e907-4780-96c1-cefc5138a2bd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:17.433136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:17.458586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:17.468583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ca7dc-23c6-4d6d-bb72-6d24d24f9676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:17.469184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ca7dc-23c6-4d6d-bb72-6d24d24f9676 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:17.469467 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ca7dc-23c6-4d6d-bb72-6d24d24f9676 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:17.469767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ca7dc-23c6-4d6d-bb72-6d24d24f9676 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:17.470494 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ca7dc-23c6-4d6d-bb72-6d24d24f9676 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:17.470804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ca7dc-23c6-4d6d-bb72-6d24d24f9676 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:17.472227 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ca7dc-23c6-4d6d-bb72-6d24d24f9676 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.489552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 256/514] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Aug 31 11:30:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:17.490161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ca7dc-23c6-4d6d-bb72-6d24d24f9676 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:17.504540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324098f-9f1d-4b99-97a3-04e4d0937f31 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:17.538718 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3d6a3-69c0-416e-9b0e-56c3f9f122a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:17.539335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3d6a3-69c0-416e-9b0e-56c3f9f122a6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:17.539615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3d6a3-69c0-416e-9b0e-56c3f9f122a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:17.539884 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3d6a3-69c0-416e-9b0e-56c3f9f122a6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:17.547691 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3d6a3-69c0-416e-9b0e-56c3f9f122a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:17.549205 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3d6a3-69c0-416e-9b0e-56c3f9f122a6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-78c3d6a3-69c0-416e-9b0e-56c3f9f122a6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:17.562840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:17.563890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 259/515] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:17.564802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3d6a3-69c0-416e-9b0e-56c3f9f122a6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:17.592127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ca7dc-23c6-4d6d-bb72-6d24d24f9676 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:17.626189 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954b116-53ec-4567-b849-381773bbda31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:17.626779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954b116-53ec-4567-b849-381773bbda31 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:17.627064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954b116-53ec-4567-b849-381773bbda31 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:17.627339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954b116-53ec-4567-b849-381773bbda31 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:17.628062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954b116-53ec-4567-b849-381773bbda31 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:17.628370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954b116-53ec-4567-b849-381773bbda31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:17.629839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954b116-53ec-4567-b849-381773bbda31 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.647004 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 257/516] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Aug 31 11:30:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:17.648016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954b116-53ec-4567-b849-381773bbda31 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:17.653949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:17.666175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3d6a3-69c0-416e-9b0e-56c3f9f122a6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:17.699530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:17.700948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430ca94-9161-4db9-979e-0e10b3ec010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:17.701558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430ca94-9161-4db9-979e-0e10b3ec010b None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:17.701842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430ca94-9161-4db9-979e-0e10b3ec010b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:17.702115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430ca94-9161-4db9-979e-0e10b3ec010b None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:17.702817 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430ca94-9161-4db9-979e-0e10b3ec010b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:17.704281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430ca94-9161-4db9-979e-0e10b3ec010b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.725038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 260/517] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:30:17 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:30:17.725776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430ca94-9161-4db9-979e-0e10b3ec010b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:17.764140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954b116-53ec-4567-b849-381773bbda31 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:17.815328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb75f2-763a-4839-bca3-37441d486c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:17.816728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb75f2-763a-4839-bca3-37441d486c4e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:17.817013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb75f2-763a-4839-bca3-37441d486c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:17.817291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb75f2-763a-4839-bca3-37441d486c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:17.829870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb75f2-763a-4839-bca3-37441d486c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:17.831370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb75f2-763a-4839-bca3-37441d486c4e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-72cb75f2-763a-4839-bca3-37441d486c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:17.847973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:17.849064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:17.850136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:17.850136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:17.850136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:17.853716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430ca94-9161-4db9-979e-0e10b3ec010b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:17.854148 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 258/518] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:17.855291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb75f2-763a-4839-bca3-37441d486c4e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:17.899155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97bc9d-006e-47d2-9ba1-4f96cc73d953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:17.899487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97bc9d-006e-47d2-9ba1-4f96cc73d953 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:17.899736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97bc9d-006e-47d2-9ba1-4f96cc73d953 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:17.900007 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97bc9d-006e-47d2-9ba1-4f96cc73d953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:17.920987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:17.980178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:17.981091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97bc9d-006e-47d2-9ba1-4f96cc73d953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:17.982626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97bc9d-006e-47d2-9ba1-4f96cc73d953 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.989032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb75f2-763a-4839-bca3-37441d486c4e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:18.026497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbb85e-7114-44a6-823b-4ea3464d8a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:18.027094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb85e-7114-44a6-823b-4ea3464d8a8f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:18.027375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb85e-7114-44a6-823b-4ea3464d8a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:18.027629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb85e-7114-44a6-823b-4ea3464d8a8f None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:18.028358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbb85e-7114-44a6-823b-4ea3464d8a8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:18.028687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbb85e-7114-44a6-823b-4ea3464d8a8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:18.030102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbb85e-7114-44a6-823b-4ea3464d8a8f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:18.048540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 259/519] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Aug 31 11:30:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:18.049311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbb85e-7114-44a6-823b-4ea3464d8a8f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:18.104932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 261/520] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:18.104932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97bc9d-006e-47d2-9ba1-4f96cc73d953 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:18.153227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:18.160343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbb85e-7114-44a6-823b-4ea3464d8a8f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:18.201989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f5e6b-0a5c-4c49-9564-0de1910f82cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:18.202875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f5e6b-0a5c-4c49-9564-0de1910f82cd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:18.203219 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f5e6b-0a5c-4c49-9564-0de1910f82cd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:18.203503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f5e6b-0a5c-4c49-9564-0de1910f82cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:18.210335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97bc9d-006e-47d2-9ba1-4f96cc73d953 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:18.229429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:18.250313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b1dd2-7698-46bc-a582-a3001848b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:18.250857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b1dd2-7698-46bc-a582-a3001848b058 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:18.251140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b1dd2-7698-46bc-a582-a3001848b058 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:18.251422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b1dd2-7698-46bc-a582-a3001848b058 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:18.252072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b1dd2-7698-46bc-a582-a3001848b058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:18.252384 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b1dd2-7698-46bc-a582-a3001848b058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:18.255448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b1dd2-7698-46bc-a582-a3001848b058 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:18.268734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 262/521] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:18.269515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b1dd2-7698-46bc-a582-a3001848b058 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:18.288583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f5e6b-0a5c-4c49-9564-0de1910f82cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:18.290042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f5e6b-0a5c-4c49-9564-0de1910f82cd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:18.381439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b1dd2-7698-46bc-a582-a3001848b058 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:18.384540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:18.405093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 260/522] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:18.405093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f5e6b-0a5c-4c49-9564-0de1910f82cd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:18.426631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78916b5-ac24-486c-b92f-0d43b7c7d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:18.427254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78916b5-ac24-486c-b92f-0d43b7c7d350 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:18.427532 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78916b5-ac24-486c-b92f-0d43b7c7d350 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:18.427801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78916b5-ac24-486c-b92f-0d43b7c7d350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:18.461946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:18.502668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78916b5-ac24-486c-b92f-0d43b7c7d350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:18.504229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78916b5-ac24-486c-b92f-0d43b7c7d350 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:18.514815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f5e6b-0a5c-4c49-9564-0de1910f82cd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:18.548914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18368da-45a6-4a28-9ce4-10e0be4ada2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:18.549532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18368da-45a6-4a28-9ce4-10e0be4ada2c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:18.549796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18368da-45a6-4a28-9ce4-10e0be4ada2c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:18.550067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18368da-45a6-4a28-9ce4-10e0be4ada2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:18.560377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18368da-45a6-4a28-9ce4-10e0be4ada2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:18.561834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18368da-45a6-4a28-9ce4-10e0be4ada2c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d18368da-45a6-4a28-9ce4-10e0be4ada2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:18.577912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:18.578801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:18.579643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:18.579643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:18.579643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:18.579643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 261/523] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:18.580126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18368da-45a6-4a28-9ce4-10e0be4ada2c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:18.614774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 263/524] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:18.615602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78916b5-ac24-486c-b92f-0d43b7c7d350 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:18.619666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:18.700680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18368da-45a6-4a28-9ce4-10e0be4ada2c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:18.703749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:18.724150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78916b5-ac24-486c-b92f-0d43b7c7d350 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:18.739345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99e11f-1b72-4d1c-8456-dfd24b403b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:18.740103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e11f-1b72-4d1c-8456-dfd24b403b69 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:18.740260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e11f-1b72-4d1c-8456-dfd24b403b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:18.740541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e11f-1b72-4d1c-8456-dfd24b403b69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:18.748533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99e11f-1b72-4d1c-8456-dfd24b403b69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:18.749949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99e11f-1b72-4d1c-8456-dfd24b403b69 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:18.761659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63374467-fdf6-4ed5-a40a-77fb8d13a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cd99e11f-1b72-4d1c-8456-dfd24b403b69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:18.762079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:18.762927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:18.763809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:18.763809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:18.763809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:18.763809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:18.763809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:18.763809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:18.763809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:18.763809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:18.764220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63374467-fdf6-4ed5-a40a-77fb8d13a0bf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:18.764220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63374467-fdf6-4ed5-a40a-77fb8d13a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:18.764220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63374467-fdf6-4ed5-a40a-77fb8d13a0bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:18.764390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 262/525] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:18.768481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99e11f-1b72-4d1c-8456-dfd24b403b69 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:18.774758 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63374467-fdf6-4ed5-a40a-77fb8d13a0bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:18.776961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63374467-fdf6-4ed5-a40a-77fb8d13a0bf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-63374467-fdf6-4ed5-a40a-77fb8d13a0bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:18.792346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:18.793390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:18.794267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:18.794267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:18.794267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:18.794267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 264/526] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:18.794858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63374467-fdf6-4ed5-a40a-77fb8d13a0bf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:18.865014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:18.887060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99e11f-1b72-4d1c-8456-dfd24b403b69 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:18.914287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63374467-fdf6-4ed5-a40a-77fb8d13a0bf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:18.926218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49732fbf-70bd-4755-bff9-1d600cc6c3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:18.926823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732fbf-70bd-4755-bff9-1d600cc6c3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:18.927101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732fbf-70bd-4755-bff9-1d600cc6c3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:18.927367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732fbf-70bd-4755-bff9-1d600cc6c3e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:18.928003 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49732fbf-70bd-4755-bff9-1d600cc6c3e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:18.928302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49732fbf-70bd-4755-bff9-1d600cc6c3e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:18.929691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49732fbf-70bd-4755-bff9-1d600cc6c3e5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:18.942813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 263/527] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:18.943865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49732fbf-70bd-4755-bff9-1d600cc6c3e5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:18.958058 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b05d2-cdb6-42b0-bf30-d079365c9a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:18.958840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b05d2-cdb6-42b0-bf30-d079365c9a1c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:18.959459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b05d2-cdb6-42b0-bf30-d079365c9a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:18.959747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b05d2-cdb6-42b0-bf30-d079365c9a1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:18.969760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:18.974369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b05d2-cdb6-42b0-bf30-d079365c9a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:18.976279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b05d2-cdb6-42b0-bf30-d079365c9a1c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-860b05d2-cdb6-42b0-bf30-d079365c9a1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:18.994364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:18.995588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:19.001315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:19.001315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:19.001315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:19.001315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 265/528] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:19.001315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b05d2-cdb6-42b0-bf30-d079365c9a1c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.059650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49732fbf-70bd-4755-bff9-1d600cc6c3e5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:19.095908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aade07-1e88-45bc-99f1-b9b76c27a52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:19.096511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aade07-1e88-45bc-99f1-b9b76c27a52f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.096808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aade07-1e88-45bc-99f1-b9b76c27a52f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.097079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aade07-1e88-45bc-99f1-b9b76c27a52f None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.097756 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aade07-1e88-45bc-99f1-b9b76c27a52f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:19.098063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aade07-1e88-45bc-99f1-b9b76c27a52f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:19.099478 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aade07-1e88-45bc-99f1-b9b76c27a52f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:19.110451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b05d2-cdb6-42b0-bf30-d079365c9a1c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:19.118263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 264/529] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:19.118924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aade07-1e88-45bc-99f1-b9b76c27a52f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.125748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:19.163434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f08f75-2cc5-48fc-b2bf-154569082601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:19.164039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f08f75-2cc5-48fc-b2bf-154569082601 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.164319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f08f75-2cc5-48fc-b2bf-154569082601 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.165106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f08f75-2cc5-48fc-b2bf-154569082601 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.165758 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f08f75-2cc5-48fc-b2bf-154569082601 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:19.166064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f08f75-2cc5-48fc-b2bf-154569082601 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:19.168115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f08f75-2cc5-48fc-b2bf-154569082601 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:19.187482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 266/530] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:19.187482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f08f75-2cc5-48fc-b2bf-154569082601 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.216801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:19.237699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aade07-1e88-45bc-99f1-b9b76c27a52f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:19.275686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64666c7b-2791-4f98-b62b-00a3851dd000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:19.276428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64666c7b-2791-4f98-b62b-00a3851dd000 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.276672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64666c7b-2791-4f98-b62b-00a3851dd000 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.276941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64666c7b-2791-4f98-b62b-00a3851dd000 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.283794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64666c7b-2791-4f98-b62b-00a3851dd000 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:19.285258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64666c7b-2791-4f98-b62b-00a3851dd000 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:19.296924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f08f75-2cc5-48fc-b2bf-154569082601 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-64666c7b-2791-4f98-b62b-00a3851dd000 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:19.297401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:19.298204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:19.299060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:19.299060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:19.299060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:19.299060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:19.299060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:19.299060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:19.299060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:19.299060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:19.299060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 265/531] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:19.299516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64666c7b-2791-4f98-b62b-00a3851dd000 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.340753 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17f976-cb49-43ee-a29b-87142b643411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:19.341767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17f976-cb49-43ee-a29b-87142b643411 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.342055 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17f976-cb49-43ee-a29b-87142b643411 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.342326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17f976-cb49-43ee-a29b-87142b643411 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.342968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17f976-cb49-43ee-a29b-87142b643411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:19.343266 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17f976-cb49-43ee-a29b-87142b643411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:19.345981 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17f976-cb49-43ee-a29b-87142b643411 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:19.358995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 267/532] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:19.359641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17f976-cb49-43ee-a29b-87142b643411 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.389271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:19.412431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64666c7b-2791-4f98-b62b-00a3851dd000 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:19.448212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653da9b-d8b3-439f-9d81-9ac7b1261175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:19.448881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653da9b-d8b3-439f-9d81-9ac7b1261175 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.449157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653da9b-d8b3-439f-9d81-9ac7b1261175 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.449431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653da9b-d8b3-439f-9d81-9ac7b1261175 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.450148 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653da9b-d8b3-439f-9d81-9ac7b1261175 None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=f4048316ea784c389eb249b4dd915e0e) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:19.450454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653da9b-d8b3-439f-9d81-9ac7b1261175 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:19.452157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653da9b-d8b3-439f-9d81-9ac7b1261175 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:19.461573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 266/533] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:19 2022] GET /identity/v3/role_assignments?user.id=f4048316ea784c389eb249b4dd915e0e&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:19.462789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653da9b-d8b3-439f-9d81-9ac7b1261175 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.484863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17f976-cb49-43ee-a29b-87142b643411 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:19.488605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:19.533363 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48ec58-24f8-4bec-899a-d7afe3aa5851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:19.534196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48ec58-24f8-4bec-899a-d7afe3aa5851 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.534571 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48ec58-24f8-4bec-899a-d7afe3aa5851 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.534939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48ec58-24f8-4bec-899a-d7afe3aa5851 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.546033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48ec58-24f8-4bec-899a-d7afe3aa5851 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:19.548293 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48ec58-24f8-4bec-899a-d7afe3aa5851 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-df48ec58-24f8-4bec-899a-d7afe3aa5851 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:19.567794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:19.568895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:19.569783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:19.569783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:19.569783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:19.569783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:19.569783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:19.569783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:19.569783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:19.569783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:19.573925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 268/534] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:19.575077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48ec58-24f8-4bec-899a-d7afe3aa5851 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.610542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653da9b-d8b3-439f-9d81-9ac7b1261175 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:19.664653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690f5e2-d034-4b08-9457-b26fe1cf89ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:19.665391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690f5e2-d034-4b08-9457-b26fe1cf89ca None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.665674 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690f5e2-d034-4b08-9457-b26fe1cf89ca None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.665948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690f5e2-d034-4b08-9457-b26fe1cf89ca None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.666901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690f5e2-d034-4b08-9457-b26fe1cf89ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:19.667101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690f5e2-d034-4b08-9457-b26fe1cf89ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:19.668548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690f5e2-d034-4b08-9457-b26fe1cf89ca None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:19.687813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 267/535] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:19.688790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690f5e2-d034-4b08-9457-b26fe1cf89ca None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.704887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:19.719053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48ec58-24f8-4bec-899a-d7afe3aa5851 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:19.757384 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcc98-5acd-46cc-b567-20e24316b431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:19.758172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcc98-5acd-46cc-b567-20e24316b431 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.758459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcc98-5acd-46cc-b567-20e24316b431 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.759048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcc98-5acd-46cc-b567-20e24316b431 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.766477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:19.769031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebcc98-5acd-46cc-b567-20e24316b431 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:19.770784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebcc98-5acd-46cc-b567-20e24316b431 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-25ebcc98-5acd-46cc-b567-20e24316b431 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:19.786909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:19.787798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:19.788685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:19.788685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:19.788685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:19.788685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:19.788685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:19.788685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:19.788685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:19.788685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:19.788685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 269/536] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:19.789138 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcc98-5acd-46cc-b567-20e24316b431 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.806247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690f5e2-d034-4b08-9457-b26fe1cf89ca None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:19.844412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa80a4-315a-49c3-abae-d0699f6b9720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:19.845166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa80a4-315a-49c3-abae-d0699f6b9720 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.845446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa80a4-315a-49c3-abae-d0699f6b9720 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.845837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa80a4-315a-49c3-abae-d0699f6b9720 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.846524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa80a4-315a-49c3-abae-d0699f6b9720 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:19.846830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa80a4-315a-49c3-abae-d0699f6b9720 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:19.848253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa80a4-315a-49c3-abae-d0699f6b9720 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:19.868603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 268/537] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:19.868603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dad92-6d29-483e-ab82-3cdd5d533f95 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.868603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dad92-6d29-483e-ab82-3cdd5d533f95 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.868603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dad92-6d29-483e-ab82-3cdd5d533f95 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.868603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 269/538] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:19.868603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dad92-6d29-483e-ab82-3cdd5d533f95 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.907673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcc98-5acd-46cc-b567-20e24316b431 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:19.944019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebc9e4-0a49-4d8d-83cf-c7757b59cfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:19.944678 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc9e4-0a49-4d8d-83cf-c7757b59cfda None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.944962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc9e4-0a49-4d8d-83cf-c7757b59cfda None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.945238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc9e4-0a49-4d8d-83cf-c7757b59cfda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.945984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebc9e4-0a49-4d8d-83cf-c7757b59cfda None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=e8dec99551ff4c31a17247cde3515acd) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:19.946274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebc9e4-0a49-4d8d-83cf-c7757b59cfda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:19.948246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebc9e4-0a49-4d8d-83cf-c7757b59cfda None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:19.964519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 270/539] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:19 2022] GET /identity/v3/role_assignments?user.id=e8dec99551ff4c31a17247cde3515acd&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:19.964519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebc9e4-0a49-4d8d-83cf-c7757b59cfda None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.978946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:20.001930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dad92-6d29-483e-ab82-3cdd5d533f95 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:20.040399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:20.053040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe8d88-b792-4619-8320-360a65f8e36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:20.054008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8d88-b792-4619-8320-360a65f8e36c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:20.054390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8d88-b792-4619-8320-360a65f8e36c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:20.055624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8d88-b792-4619-8320-360a65f8e36c None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:20.056340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe8d88-b792-4619-8320-360a65f8e36c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:20.056673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe8d88-b792-4619-8320-360a65f8e36c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:20.058251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe8d88-b792-4619-8320-360a65f8e36c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:20.077704 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 270/540] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:20.079117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4ac9-d254-4a3a-93aa-7b781757c68c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:20.079406 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4ac9-d254-4a3a-93aa-7b781757c68c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:20.079679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4ac9-d254-4a3a-93aa-7b781757c68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:20.102499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebc9e4-0a49-4d8d-83cf-c7757b59cfda None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:20.111108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-46aa4ac9-d254-4a3a-93aa-7b781757c68c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:20.112417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-46aa4ac9-d254-4a3a-93aa-7b781757c68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:20.153561 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e22db-2cac-431b-b8fb-42688c7389a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:20.154194 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e22db-2cac-431b-b8fb-42688c7389a3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:20.154472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e22db-2cac-431b-b8fb-42688c7389a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:20.154744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e22db-2cac-431b-b8fb-42688c7389a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:20.155738 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e22db-2cac-431b-b8fb-42688c7389a3 None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=30546ba7f5614f878a9ba322c7c55ccb) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:20.156045 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e22db-2cac-431b-b8fb-42688c7389a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:20.158467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e22db-2cac-431b-b8fb-42688c7389a3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:20.173785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 271/541] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:19 2022] GET /identity/v3/role_assignments?user.id=30546ba7f5614f878a9ba322c7c55ccb&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:20.174794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e22db-2cac-431b-b8fb-42688c7389a3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:20.224380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 271/542] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:20.297093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:20.327765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e22db-2cac-431b-b8fb-42688c7389a3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:20.392090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cae2d4-3810-4822-9856-331ed1c4ea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:20.392732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cae2d4-3810-4822-9856-331ed1c4ea8f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:20.393017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cae2d4-3810-4822-9856-331ed1c4ea8f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:20.393296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cae2d4-3810-4822-9856-331ed1c4ea8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:20.394032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cae2d4-3810-4822-9856-331ed1c4ea8f None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=18ec048788904bc58245e878a8644091) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:20.394342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cae2d4-3810-4822-9856-331ed1c4ea8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:20.395874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cae2d4-3810-4822-9856-331ed1c4ea8f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:20.408807 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 272/543] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:20 2022] GET /identity/v3/role_assignments?user.id=18ec048788904bc58245e878a8644091&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:20.528325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa4ac9-d254-4a3a-93aa-7b781757c68c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:20.658440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa4ac9-d254-4a3a-93aa-7b781757c68c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:20.684957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:20.700502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af3691-cf23-453f-8fc1-40830a8a1553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:20.701150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af3691-cf23-453f-8fc1-40830a8a1553 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:20.704518 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af3691-cf23-453f-8fc1-40830a8a1553 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:20.704518 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af3691-cf23-453f-8fc1-40830a8a1553 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:20.712701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af3691-cf23-453f-8fc1-40830a8a1553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c7af3691-cf23-453f-8fc1-40830a8a1553 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:20.718042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:20.718907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:20.719722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 272/544] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:20.725003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cae2d4-3810-4822-9856-331ed1c4ea8f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:20.943900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cae2d4-3810-4822-9856-331ed1c4ea8f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:20.996061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf97b0-4738-48da-bc46-7cbddfa489c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:20.996739 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf97b0-4738-48da-bc46-7cbddfa489c6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:20.996961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf97b0-4738-48da-bc46-7cbddfa489c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:20.997225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf97b0-4738-48da-bc46-7cbddfa489c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:20.997887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf97b0-4738-48da-bc46-7cbddfa489c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:20.998188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf97b0-4738-48da-bc46-7cbddfa489c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:20.999555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf97b0-4738-48da-bc46-7cbddfa489c6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:21.016707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 273/545] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:21.017648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af3691-cf23-453f-8fc1-40830a8a1553 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:21.201399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af3691-cf23-453f-8fc1-40830a8a1553 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:21.277212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aed48-4fa0-4d3b-a820-64348cd30dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:21.278255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aed48-4fa0-4d3b-a820-64348cd30dbb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:21.278748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aed48-4fa0-4d3b-a820-64348cd30dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:21.279220 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aed48-4fa0-4d3b-a820-64348cd30dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:21.296001 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154aed48-4fa0-4d3b-a820-64348cd30dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:21.297769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154aed48-4fa0-4d3b-a820-64348cd30dbb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-154aed48-4fa0-4d3b-a820-64348cd30dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:21.317729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:21.318676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:21.319515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:21.319515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:21.319515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:21.321601 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 273/546] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:21.329667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf97b0-4738-48da-bc46-7cbddfa489c6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:21.348524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:21.472340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf97b0-4738-48da-bc46-7cbddfa489c6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:21.531666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6587efa-d0c3-42a1-af50-e453ac791b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:21.532279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6587efa-d0c3-42a1-af50-e453ac791b7a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:21.532581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6587efa-d0c3-42a1-af50-e453ac791b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:21.532857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6587efa-d0c3-42a1-af50-e453ac791b7a None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:21.533507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6587efa-d0c3-42a1-af50-e453ac791b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:21.533806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6587efa-d0c3-42a1-af50-e453ac791b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:21.535178 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6587efa-d0c3-42a1-af50-e453ac791b7a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:21.553851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 274/547] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:21.559713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aed48-4fa0-4d3b-a820-64348cd30dbb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:21.623058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:21.640677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6b9b-06ba-46a5-98e8-2fb2289131af None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:21.640962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6b9b-06ba-46a5-98e8-2fb2289131af None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:21.641243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6b9b-06ba-46a5-98e8-2fb2289131af None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:21.642923 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 275/548] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:21.665551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af0953-bbe4-47c5-845a-673c6ec8d50c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:21.665856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af0953-bbe4-47c5-845a-673c6ec8d50c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:21.666115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af0953-bbe4-47c5-845a-673c6ec8d50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:21.700088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b3af0953-bbe4-47c5-845a-673c6ec8d50c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:21.703032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b3af0953-bbe4-47c5-845a-673c6ec8d50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:21.758419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aed48-4fa0-4d3b-a820-64348cd30dbb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:21.822505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c87c1-41b6-488c-9f09-b2549f63811c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:21.823691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c87c1-41b6-488c-9f09-b2549f63811c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:21.823980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c87c1-41b6-488c-9f09-b2549f63811c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:21.824222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c87c1-41b6-488c-9f09-b2549f63811c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:21.832440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c87c1-41b6-488c-9f09-b2549f63811c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:21.834083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c87c1-41b6-488c-9f09-b2549f63811c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f30c87c1-41b6-488c-9f09-b2549f63811c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:21.845686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:21.847080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:21.848135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:21.848135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:21.848135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:21.848135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:21.848135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:21.848135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:21.848135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:21.848135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:21.848135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:21.848135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:21.848135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 274/549] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:21.852262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c87c1-41b6-488c-9f09-b2549f63811c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:21.853313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:21.875095 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 276/550] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 11:30:22.019300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:22.054746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c87c1-41b6-488c-9f09-b2549f63811c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:22.112278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b46e12-49c8-40e0-a773-81873a4fa77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:22.113008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b46e12-49c8-40e0-a773-81873a4fa77e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:22.113341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b46e12-49c8-40e0-a773-81873a4fa77e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:22.113613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b46e12-49c8-40e0-a773-81873a4fa77e None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:22.114392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b46e12-49c8-40e0-a773-81873a4fa77e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:22.114697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b46e12-49c8-40e0-a773-81873a4fa77e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:22.116202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b46e12-49c8-40e0-a773-81873a4fa77e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:22.138739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 275/551] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Aug 31 11:30:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:22.144835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af0953-bbe4-47c5-845a-673c6ec8d50c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:22.271219 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b46e12-49c8-40e0-a773-81873a4fa77e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:22.313986 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af0953-bbe4-47c5-845a-673c6ec8d50c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:22.384784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb61ec-d943-48fb-ae9a-b057dc6375b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:22.386565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb61ec-d943-48fb-ae9a-b057dc6375b9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:22.387723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb61ec-d943-48fb-ae9a-b057dc6375b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:22.388388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb61ec-d943-48fb-ae9a-b057dc6375b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:22.444662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b46e12-49c8-40e0-a773-81873a4fa77e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:22.473206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:22.485958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb61ec-d943-48fb-ae9a-b057dc6375b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:22.487731 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb61ec-d943-48fb-ae9a-b057dc6375b9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:22.498506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c2372-7c3d-498f-bdc6-478473a8a767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:22.499155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c2372-7c3d-498f-bdc6-478473a8a767 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:22.499435 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c2372-7c3d-498f-bdc6-478473a8a767 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:22.499708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c2372-7c3d-498f-bdc6-478473a8a767 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:22.512414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c2372-7c3d-498f-bdc6-478473a8a767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f37c2372-7c3d-498f-bdc6-478473a8a767 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:22.517699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:22.518706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:22.519529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:22.523899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 276/552] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:22.525340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4912d-4029-42d4-9713-8722d46e6028 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:22.525625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4912d-4029-42d4-9713-8722d46e6028 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:22.526450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4912d-4029-42d4-9713-8722d46e6028 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:22.532586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 277/553] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:22.533399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4912d-4029-42d4-9713-8722d46e6028 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:22.545927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:22.621897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 277/554] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:22.621897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b21c6c-ff9a-49bd-aced-1948d8d9e4a2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:22.621897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b21c6c-ff9a-49bd-aced-1948d8d9e4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:22.621897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b21c6c-ff9a-49bd-aced-1948d8d9e4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:22.638485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-18b21c6c-ff9a-49bd-aced-1948d8d9e4a2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:22.639656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-18b21c6c-ff9a-49bd-aced-1948d8d9e4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:22.651515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4912d-4029-42d4-9713-8722d46e6028 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:22.689542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f6b0b-1343-4cb6-b5bf-d1df5b87ea58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:22.690147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f6b0b-1343-4cb6-b5bf-d1df5b87ea58 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:22.690427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f6b0b-1343-4cb6-b5bf-d1df5b87ea58 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:22.690698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f6b0b-1343-4cb6-b5bf-d1df5b87ea58 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:22.691616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f6b0b-1343-4cb6-b5bf-d1df5b87ea58 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:22.691919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f6b0b-1343-4cb6-b5bf-d1df5b87ea58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:22.693311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f6b0b-1343-4cb6-b5bf-d1df5b87ea58 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:22.702551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 278/555] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:22.705898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327e548-fb6f-4ea7-ac79-1edf5b13a923 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:22.707193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327e548-fb6f-4ea7-ac79-1edf5b13a923 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:22.707464 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327e548-fb6f-4ea7-ac79-1edf5b13a923 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:22.708892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 279/556] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:22.713959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327e548-fb6f-4ea7-ac79-1edf5b13a923 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:22.717833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 278/557] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:22.718681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b21c6c-ff9a-49bd-aced-1948d8d9e4a2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:22.745494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:22.788222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:22.824323 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327e548-fb6f-4ea7-ac79-1edf5b13a923 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:22.829885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b21c6c-ff9a-49bd-aced-1948d8d9e4a2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:22.859018 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3badd9-817c-4bb6-b6ff-d888b9a5330e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:22.859635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3badd9-817c-4bb6-b6ff-d888b9a5330e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:22.859917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3badd9-817c-4bb6-b6ff-d888b9a5330e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:22.860248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3badd9-817c-4bb6-b6ff-d888b9a5330e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:22.865375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609882e-d70b-4f18-912f-fe4053cccfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:22.865989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609882e-d70b-4f18-912f-fe4053cccfdf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:22.866270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609882e-d70b-4f18-912f-fe4053cccfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:22.866648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609882e-d70b-4f18-912f-fe4053cccfdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:22.870888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3badd9-817c-4bb6-b6ff-d888b9a5330e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:22.872312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3badd9-817c-4bb6-b6ff-d888b9a5330e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:22.877213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609882e-d70b-4f18-912f-fe4053cccfdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:22.878639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609882e-d70b-4f18-912f-fe4053cccfdf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5d3badd9-817c-4bb6-b6ff-d888b9a5330e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:22.887333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:22.888276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:22.889523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:22.889523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:22.889523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:22.889523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 280/558] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:22.889841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b3cd6-51b0-49a6-93e5-6759708cb77b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:22.889841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b3cd6-51b0-49a6-93e5-6759708cb77b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:22.890418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b3cd6-51b0-49a6-93e5-6759708cb77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-2609882e-d70b-4f18-912f-fe4053cccfdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:22.899024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:22.899994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:22.901067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:22.901067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:22.901067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:22.901067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 279/559] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:22.902262 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b35d5-7341-4f42-b97f-7cd8012485ab None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:22.902605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b35d5-7341-4f42-b97f-7cd8012485ab None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:22.902944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b35d5-7341-4f42-b97f-7cd8012485ab None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:22.904376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 280/560] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:22.908865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b35d5-7341-4f42-b97f-7cd8012485ab None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:22.915581 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e34b3cd6-51b0-49a6-93e5-6759708cb77b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:22.915721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e34b3cd6-51b0-49a6-93e5-6759708cb77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:22.988248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 281/561] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:22.989120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b3cd6-51b0-49a6-93e5-6759708cb77b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:22.993283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:23.022411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b35d5-7341-4f42-b97f-7cd8012485ab None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.050961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:23.062375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d933a0-1257-478a-91b2-76b0c587e82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:23.062986 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d933a0-1257-478a-91b2-76b0c587e82a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.063255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d933a0-1257-478a-91b2-76b0c587e82a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.063529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d933a0-1257-478a-91b2-76b0c587e82a None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.064168 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d933a0-1257-478a-91b2-76b0c587e82a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:23.064472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d933a0-1257-478a-91b2-76b0c587e82a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:23.065871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d933a0-1257-478a-91b2-76b0c587e82a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:23.086299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 281/562] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:23.086299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c5dce-df36-4a10-9750-f8b228c3fedf None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.086299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c5dce-df36-4a10-9750-f8b228c3fedf None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.086299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c5dce-df36-4a10-9750-f8b228c3fedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.104447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b35c5dce-df36-4a10-9750-f8b228c3fedf None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:23.105857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b35c5dce-df36-4a10-9750-f8b228c3fedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:23.106377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b3cd6-51b0-49a6-93e5-6759708cb77b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.150542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f2edd-dc8f-4ee8-b519-088c7e653a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:23.151192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f2edd-dc8f-4ee8-b519-088c7e653a80 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.151556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f2edd-dc8f-4ee8-b519-088c7e653a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.151838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f2edd-dc8f-4ee8-b519-088c7e653a80 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.152505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f2edd-dc8f-4ee8-b519-088c7e653a80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:23.152843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f2edd-dc8f-4ee8-b519-088c7e653a80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:23.154211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f2edd-dc8f-4ee8-b519-088c7e653a80 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:23.165607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 282/563] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:23.166451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f2edd-dc8f-4ee8-b519-088c7e653a80 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:23.186546 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 282/564] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:23.187955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c5dce-df36-4a10-9750-f8b228c3fedf None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:23.263119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:23.297319 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f2edd-dc8f-4ee8-b519-088c7e653a80 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.305583 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c5dce-df36-4a10-9750-f8b228c3fedf None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.311259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:23.334336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8548e6a-f0e8-493a-a9c0-74e5a1391b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:23.335260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8548e6a-f0e8-493a-a9c0-74e5a1391b5f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.335718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8548e6a-f0e8-493a-a9c0-74e5a1391b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.336182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8548e6a-f0e8-493a-a9c0-74e5a1391b5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.345573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d28a85-4183-4233-9277-61fb118c6a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:23.346292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d28a85-4183-4233-9277-61fb118c6a57 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.346580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d28a85-4183-4233-9277-61fb118c6a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.346853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d28a85-4183-4233-9277-61fb118c6a57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.347892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8548e6a-f0e8-493a-a9c0-74e5a1391b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e8548e6a-f0e8-493a-a9c0-74e5a1391b5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:23.353337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:23.354580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 283/565] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:23.355409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8548e6a-f0e8-493a-a9c0-74e5a1391b5f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:23.360016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d28a85-4183-4233-9277-61fb118c6a57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:23.362412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d28a85-4183-4233-9277-61fb118c6a57 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-16d28a85-4183-4233-9277-61fb118c6a57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:23.377342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:23.378243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:23.379106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:23.379106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:23.379106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:23.379106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:23.379106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:23.379106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:23.379106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:23.379106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:23.383442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 283/566] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:23.383776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d28a85-4183-4233-9277-61fb118c6a57 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:23.464959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8548e6a-f0e8-493a-a9c0-74e5a1391b5f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.488759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d28a85-4183-4233-9277-61fb118c6a57 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.498984 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e58edc-7dc0-4308-83f2-a31683b36953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:23.499599 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e58edc-7dc0-4308-83f2-a31683b36953 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.499878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e58edc-7dc0-4308-83f2-a31683b36953 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.500150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e58edc-7dc0-4308-83f2-a31683b36953 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.507475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e58edc-7dc0-4308-83f2-a31683b36953 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:23.508921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e58edc-7dc0-4308-83f2-a31683b36953 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:23.511649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-90e58edc-7dc0-4308-83f2-a31683b36953 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:23.520386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:23.521392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:23.522327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:23.522327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:23.522327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:23.522327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:23.522327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:23.522327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:23.522327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:23.522327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:23.522327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:23.522327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:23.522327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 284/567] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Aug 31 11:30:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:23.522910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e58edc-7dc0-4308-83f2-a31683b36953 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:23.528601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113223e-4902-4e6a-97ff-f521d21a809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:23.529232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113223e-4902-4e6a-97ff-f521d21a809b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.529511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113223e-4902-4e6a-97ff-f521d21a809b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.529785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113223e-4902-4e6a-97ff-f521d21a809b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.539871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113223e-4902-4e6a-97ff-f521d21a809b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-e113223e-4902-4e6a-97ff-f521d21a809b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:23.545400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:23.546531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 284/568] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:23.547564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113223e-4902-4e6a-97ff-f521d21a809b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:23.559562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:23.629044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e58edc-7dc0-4308-83f2-a31683b36953 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.648215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113223e-4902-4e6a-97ff-f521d21a809b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.661687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190c5b-ec4c-4c0c-8c60-06951725a0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:23.662293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190c5b-ec4c-4c0c-8c60-06951725a0cf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.662576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190c5b-ec4c-4c0c-8c60-06951725a0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.663510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190c5b-ec4c-4c0c-8c60-06951725a0cf None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.663510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70190c5b-ec4c-4c0c-8c60-06951725a0cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:23.663799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70190c5b-ec4c-4c0c-8c60-06951725a0cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:23.665198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70190c5b-ec4c-4c0c-8c60-06951725a0cf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:23.674646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 285/569] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:23.675549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190c5b-ec4c-4c0c-8c60-06951725a0cf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:23.688066 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ee99c-7f8e-49c7-9cee-69fb054b28c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:23.688718 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ee99c-7f8e-49c7-9cee-69fb054b28c7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.689003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ee99c-7f8e-49c7-9cee-69fb054b28c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.689278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ee99c-7f8e-49c7-9cee-69fb054b28c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.689980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ee99c-7f8e-49c7-9cee-69fb054b28c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:23.690296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ee99c-7f8e-49c7-9cee-69fb054b28c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:23.691984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ee99c-7f8e-49c7-9cee-69fb054b28c7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:23.709625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 285/570] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:23.710545 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ee99c-7f8e-49c7-9cee-69fb054b28c7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:23.756346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:23.784975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:23.796887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190c5b-ec4c-4c0c-8c60-06951725a0cf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.816860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ee99c-7f8e-49c7-9cee-69fb054b28c7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.831254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52504ce-79c8-4c65-b1ed-c911ff19607d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:23.831880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52504ce-79c8-4c65-b1ed-c911ff19607d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.832156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52504ce-79c8-4c65-b1ed-c911ff19607d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.832433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52504ce-79c8-4c65-b1ed-c911ff19607d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.842629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52504ce-79c8-4c65-b1ed-c911ff19607d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f52504ce-79c8-4c65-b1ed-c911ff19607d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:23.847888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:23.849030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 286/571] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:23.850002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52504ce-79c8-4c65-b1ed-c911ff19607d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:23.855041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae372-89ca-47ee-904d-d775eeec3b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:23.855731 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae372-89ca-47ee-904d-d775eeec3b90 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.856008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae372-89ca-47ee-904d-d775eeec3b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.856277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae372-89ca-47ee-904d-d775eeec3b90 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.857044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ae372-89ca-47ee-904d-d775eeec3b90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:23.858113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ae372-89ca-47ee-904d-d775eeec3b90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:23.858764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ae372-89ca-47ee-904d-d775eeec3b90 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:23.896392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 286/572] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Aug 31 11:30:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:23.897294 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae372-89ca-47ee-904d-d775eeec3b90 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:23.956598 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52504ce-79c8-4c65-b1ed-c911ff19607d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.991148 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18826147-a10c-4090-bddb-fc00e1eeef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:23.991835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18826147-a10c-4090-bddb-fc00e1eeef80 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:23.992178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18826147-a10c-4090-bddb-fc00e1eeef80 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:23.992540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18826147-a10c-4090-bddb-fc00e1eeef80 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:23.993200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18826147-a10c-4090-bddb-fc00e1eeef80 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:23.993500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18826147-a10c-4090-bddb-fc00e1eeef80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:23.995123 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18826147-a10c-4090-bddb-fc00e1eeef80 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:23.998339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae372-89ca-47ee-904d-d775eeec3b90 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:23.999860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:24.003475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 287/573] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:24.004477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18826147-a10c-4090-bddb-fc00e1eeef80 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:24.025036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:24.033434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2fbc8-2903-445b-a2bc-6e26bfab7b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:24.034064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2fbc8-2903-445b-a2bc-6e26bfab7b1d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:24.034344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2fbc8-2903-445b-a2bc-6e26bfab7b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:24.034620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2fbc8-2903-445b-a2bc-6e26bfab7b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:24.044979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2fbc8-2903-445b-a2bc-6e26bfab7b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:24.046519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2fbc8-2903-445b-a2bc-6e26bfab7b1d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-cff2fbc8-2903-445b-a2bc-6e26bfab7b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:24.060903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:24.061910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:24.062775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:24.062775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:24.062775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:24.062775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 287/574] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:24.063123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2fbc8-2903-445b-a2bc-6e26bfab7b1d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:24.115313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18826147-a10c-4090-bddb-fc00e1eeef80 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:24.151697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44509183-568e-43ce-add9-bb6340766e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:24.152288 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509183-568e-43ce-add9-bb6340766e15 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:24.152601 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509183-568e-43ce-add9-bb6340766e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:24.152889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509183-568e-43ce-add9-bb6340766e15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:24.153792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44509183-568e-43ce-add9-bb6340766e15 None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=3384c24dd081414bbf0a2e3d9f71126b) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:24.154097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44509183-568e-43ce-add9-bb6340766e15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:24.155523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44509183-568e-43ce-add9-bb6340766e15 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:24.161237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 288/575] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/role_assignments?user.id=3384c24dd081414bbf0a2e3d9f71126b&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:24.161939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44509183-568e-43ce-add9-bb6340766e15 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:24.172416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2fbc8-2903-445b-a2bc-6e26bfab7b1d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:24.215031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856abf1d-db45-42d9-a374-a6e412b80070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:24.215732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856abf1d-db45-42d9-a374-a6e412b80070 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:24.215945 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856abf1d-db45-42d9-a374-a6e412b80070 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:24.216203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856abf1d-db45-42d9-a374-a6e412b80070 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:24.216926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856abf1d-db45-42d9-a374-a6e412b80070 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:24.217237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856abf1d-db45-42d9-a374-a6e412b80070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:24.219384 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856abf1d-db45-42d9-a374-a6e412b80070 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:24.234673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 288/576] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:24.234673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856abf1d-db45-42d9-a374-a6e412b80070 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:24.253283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:24.272412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:24.295238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44509183-568e-43ce-add9-bb6340766e15 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:24.338906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45ce54-499b-4082-a261-e08a1c6c4423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:24.339283 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45ce54-499b-4082-a261-e08a1c6c4423 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:24.339502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45ce54-499b-4082-a261-e08a1c6c4423 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:24.339779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45ce54-499b-4082-a261-e08a1c6c4423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:24.350670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45ce54-499b-4082-a261-e08a1c6c4423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:24.352115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45ce54-499b-4082-a261-e08a1c6c4423 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:24.353567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856abf1d-db45-42d9-a374-a6e412b80070 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-da45ce54-499b-4082-a261-e08a1c6c4423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:24.367551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:24.368553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:24.369585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:24.369585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:24.369585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:24.369585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 289/577] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:24.369585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45ce54-499b-4082-a261-e08a1c6c4423 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:24.395035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f58b-1af7-4376-ae16-832db1208b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:24.395671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f58b-1af7-4376-ae16-832db1208b82 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:24.395955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f58b-1af7-4376-ae16-832db1208b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:24.396232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f58b-1af7-4376-ae16-832db1208b82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:24.407992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6f58b-1af7-4376-ae16-832db1208b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:24.409494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6f58b-1af7-4376-ae16-832db1208b82 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b3f6f58b-1af7-4376-ae16-832db1208b82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:24.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:24.427903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:24.428845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:24.428845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:24.428845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:24.433010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 289/578] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:24.433863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f58b-1af7-4376-ae16-832db1208b82 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:24.490357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45ce54-499b-4082-a261-e08a1c6c4423 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:24.504992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:24.524649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:24.531469 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c643e-1d4b-4c4c-86e9-b667f05a7f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:24.532586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c643e-1d4b-4c4c-86e9-b667f05a7f3f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:24.532879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c643e-1d4b-4c4c-86e9-b667f05a7f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:24.533154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c643e-1d4b-4c4c-86e9-b667f05a7f3f None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:24.533797 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c643e-1d4b-4c4c-86e9-b667f05a7f3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:24.534103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c643e-1d4b-4c4c-86e9-b667f05a7f3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:24.535501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c643e-1d4b-4c4c-86e9-b667f05a7f3f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:24.544296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f58b-1af7-4376-ae16-832db1208b82 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:24.549230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 290/579] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:24.554272 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c643e-1d4b-4c4c-86e9-b667f05a7f3f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:24.581647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ce6f7-e5ec-4347-9efb-08366f1e4485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:24.582278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ce6f7-e5ec-4347-9efb-08366f1e4485 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:24.582560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ce6f7-e5ec-4347-9efb-08366f1e4485 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:24.582833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ce6f7-e5ec-4347-9efb-08366f1e4485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:24.595005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ce6f7-e5ec-4347-9efb-08366f1e4485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:24.596631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ce6f7-e5ec-4347-9efb-08366f1e4485 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-224ce6f7-e5ec-4347-9efb-08366f1e4485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:24.614364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:24.615371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:24.616238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:24.616238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:24.616238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:24.616238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 290/580] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:24.616468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ce6f7-e5ec-4347-9efb-08366f1e4485 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:24.671012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c643e-1d4b-4c4c-86e9-b667f05a7f3f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:24.716988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745061b0-85ec-4bbb-a4a9-abe916b1a1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:24.717678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745061b0-85ec-4bbb-a4a9-abe916b1a1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:24.718023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745061b0-85ec-4bbb-a4a9-abe916b1a1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:24.718360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745061b0-85ec-4bbb-a4a9-abe916b1a1a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:24.719076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745061b0-85ec-4bbb-a4a9-abe916b1a1a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:24.719440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745061b0-85ec-4bbb-a4a9-abe916b1a1a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:24.720893 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745061b0-85ec-4bbb-a4a9-abe916b1a1a3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:24.732292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ce6f7-e5ec-4347-9efb-08366f1e4485 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:24.736963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 291/581] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:24.738013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745061b0-85ec-4bbb-a4a9-abe916b1a1a3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:24.755392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:24.772219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463b749-3a72-47ad-9ebf-6b94afa0f587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:24.772861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463b749-3a72-47ad-9ebf-6b94afa0f587 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:24.773144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463b749-3a72-47ad-9ebf-6b94afa0f587 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:24.773542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463b749-3a72-47ad-9ebf-6b94afa0f587 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:24.774300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463b749-3a72-47ad-9ebf-6b94afa0f587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:24.774606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463b749-3a72-47ad-9ebf-6b94afa0f587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:24.776089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463b749-3a72-47ad-9ebf-6b94afa0f587 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:24.793809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 291/582] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:24.794573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463b749-3a72-47ad-9ebf-6b94afa0f587 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:24.795820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:24.851772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745061b0-85ec-4bbb-a4a9-abe916b1a1a3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:24.891826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707248fd-c303-4be7-ab35-fdce849ce62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:24.892450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707248fd-c303-4be7-ab35-fdce849ce62e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:24.892763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707248fd-c303-4be7-ab35-fdce849ce62e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:24.893037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707248fd-c303-4be7-ab35-fdce849ce62e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:24.900325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707248fd-c303-4be7-ab35-fdce849ce62e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:24.902155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463b749-3a72-47ad-9ebf-6b94afa0f587 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:24.902720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707248fd-c303-4be7-ab35-fdce849ce62e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-707248fd-c303-4be7-ab35-fdce849ce62e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:24.914532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:24.915449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:24.916437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:24.916437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:24.916437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:24.916437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:24.916437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:24.916437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:24.916437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:24.916437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:24.916437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 11:30:24.916437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:24.916437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 292/583] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:24.917869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707248fd-c303-4be7-ab35-fdce849ce62e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:24.945767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d5435-568c-4505-8fed-cb23cd78fb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:24.946463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d5435-568c-4505-8fed-cb23cd78fb56 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:24.946858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d5435-568c-4505-8fed-cb23cd78fb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:24.947233 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d5435-568c-4505-8fed-cb23cd78fb56 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:24.948060 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d5435-568c-4505-8fed-cb23cd78fb56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:24.948467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d5435-568c-4505-8fed-cb23cd78fb56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:24.950083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d5435-568c-4505-8fed-cb23cd78fb56 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:24.962904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 292/584] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:24.964712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d5435-568c-4505-8fed-cb23cd78fb56 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:25.010469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:25.031884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707248fd-c303-4be7-ab35-fdce849ce62e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:25.045683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:25.073683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cd1dc-0bfe-4cd1-8b61-1d8d59ebd133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:25.074584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cd1dc-0bfe-4cd1-8b61-1d8d59ebd133 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:25.075041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cd1dc-0bfe-4cd1-8b61-1d8d59ebd133 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:25.075474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d5435-568c-4505-8fed-cb23cd78fb56 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:25.075711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cd1dc-0bfe-4cd1-8b61-1d8d59ebd133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:25.083567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cd1dc-0bfe-4cd1-8b61-1d8d59ebd133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:25.085228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cd1dc-0bfe-4cd1-8b61-1d8d59ebd133 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0a0cd1dc-0bfe-4cd1-8b61-1d8d59ebd133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:25.096911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:25.097803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:25.098689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:25.098689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:25.098689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:25.098689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:25.098689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:25.098689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:25.098689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:25.098689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:25.101256 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 293/585] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:25.102046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cd1dc-0bfe-4cd1-8b61-1d8d59ebd133 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:25.115497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7fe7a-73ab-43cd-a60c-234ecfe47543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:25.116135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7fe7a-73ab-43cd-a60c-234ecfe47543 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:25.116418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7fe7a-73ab-43cd-a60c-234ecfe47543 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:25.116733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7fe7a-73ab-43cd-a60c-234ecfe47543 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:25.136515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7fe7a-73ab-43cd-a60c-234ecfe47543 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:25.136515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7fe7a-73ab-43cd-a60c-234ecfe47543 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-9bc7fe7a-73ab-43cd-a60c-234ecfe47543 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:25.139270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:25.140214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:25.141230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:25.141230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:25.141230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:25.141230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:25.141230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:25.141230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:25.141230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:25.141230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:25.141230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 11:30:25.141230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:25.145681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 293/586] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:25.146400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7fe7a-73ab-43cd-a60c-234ecfe47543 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:25.224966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cd1dc-0bfe-4cd1-8b61-1d8d59ebd133 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:25.254256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:25.257995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7fe7a-73ab-43cd-a60c-234ecfe47543 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:25.265964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29113e3-2f54-4dcb-8c4e-f8f9b58ca972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:25.266577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29113e3-2f54-4dcb-8c4e-f8f9b58ca972 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:25.266861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29113e3-2f54-4dcb-8c4e-f8f9b58ca972 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:25.267137 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29113e3-2f54-4dcb-8c4e-f8f9b58ca972 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:25.267872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29113e3-2f54-4dcb-8c4e-f8f9b58ca972 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:25.268181 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29113e3-2f54-4dcb-8c4e-f8f9b58ca972 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:25.269784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29113e3-2f54-4dcb-8c4e-f8f9b58ca972 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:25.288971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 294/587] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Aug 31 11:30:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:25.289861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29113e3-2f54-4dcb-8c4e-f8f9b58ca972 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:25.300832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8b252-fd48-48e0-bb3e-0656271e4eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:25.301539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8b252-fd48-48e0-bb3e-0656271e4eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:25.301891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8b252-fd48-48e0-bb3e-0656271e4eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:25.302242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8b252-fd48-48e0-bb3e-0656271e4eb2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:25.310435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8b252-fd48-48e0-bb3e-0656271e4eb2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:25.311966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8b252-fd48-48e0-bb3e-0656271e4eb2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:25.323135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-22f8b252-fd48-48e0-bb3e-0656271e4eb2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:25.324925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:25.325906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 294/588] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:25.326838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8b252-fd48-48e0-bb3e-0656271e4eb2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:25.405761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29113e3-2f54-4dcb-8c4e-f8f9b58ca972 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:25.439621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8b252-fd48-48e0-bb3e-0656271e4eb2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:25.442811 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e16735-9bb2-4204-876b-ccaca4f303a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:25.443487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e16735-9bb2-4204-876b-ccaca4f303a4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:25.443842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e16735-9bb2-4204-876b-ccaca4f303a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:25.444190 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e16735-9bb2-4204-876b-ccaca4f303a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:25.444985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e16735-9bb2-4204-876b-ccaca4f303a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:25.445360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e16735-9bb2-4204-876b-ccaca4f303a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:25.446888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e16735-9bb2-4204-876b-ccaca4f303a4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:25.464369 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 295/589] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:25.464369 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e16735-9bb2-4204-876b-ccaca4f303a4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:25.492595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187b759-3274-4bd0-a08e-2ec64ca8d54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:25.493476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187b759-3274-4bd0-a08e-2ec64ca8d54f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:25.493865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187b759-3274-4bd0-a08e-2ec64ca8d54f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:25.494357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187b759-3274-4bd0-a08e-2ec64ca8d54f None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:25.495625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187b759-3274-4bd0-a08e-2ec64ca8d54f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:25.496128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187b759-3274-4bd0-a08e-2ec64ca8d54f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:25.498611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187b759-3274-4bd0-a08e-2ec64ca8d54f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:25.526683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 295/590] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Aug 31 11:30:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:25.528165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187b759-3274-4bd0-a08e-2ec64ca8d54f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:25.550280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:25.587943 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:25.599501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e16735-9bb2-4204-876b-ccaca4f303a4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:25.638493 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bd56a-6724-4915-9130-2ba91f8d6b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:25.639122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bd56a-6724-4915-9130-2ba91f8d6b55 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:25.639407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bd56a-6724-4915-9130-2ba91f8d6b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:25.639683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bd56a-6724-4915-9130-2ba91f8d6b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:25.640195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187b759-3274-4bd0-a08e-2ec64ca8d54f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:25.652991 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bd56a-6724-4915-9130-2ba91f8d6b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:25.654574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bd56a-6724-4915-9130-2ba91f8d6b55 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-695bd56a-6724-4915-9130-2ba91f8d6b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:25.670817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:25.671776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:25.672705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:25.672705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:25.672705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:25.672705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 296/591] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:25.673506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bd56a-6724-4915-9130-2ba91f8d6b55 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:25.685451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32ba8e-cbda-46de-a47d-53bd27c20cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:25.686146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32ba8e-cbda-46de-a47d-53bd27c20cae None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:25.688005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32ba8e-cbda-46de-a47d-53bd27c20cae None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:25.688005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32ba8e-cbda-46de-a47d-53bd27c20cae None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:25.688005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32ba8e-cbda-46de-a47d-53bd27c20cae None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:25.688005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32ba8e-cbda-46de-a47d-53bd27c20cae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:25.690044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32ba8e-cbda-46de-a47d-53bd27c20cae None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:25.709096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 296/592] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Aug 31 11:30:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:25.709986 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32ba8e-cbda-46de-a47d-53bd27c20cae None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:25.799440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bd56a-6724-4915-9130-2ba91f8d6b55 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:25.800356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:25.830638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32ba8e-cbda-46de-a47d-53bd27c20cae None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:25.839749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de3a35-d910-49ab-a650-22c4867d9600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:25.840410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de3a35-d910-49ab-a650-22c4867d9600 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:25.840738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de3a35-d910-49ab-a650-22c4867d9600 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:25.841014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de3a35-d910-49ab-a650-22c4867d9600 None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/f4048316ea784c389eb249b4dd915e0e/roles/babbae9ac71c4c15817721af5d270a80` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:25.854154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:25.857750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de3a35-d910-49ab-a650-22c4867d9600 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=f4048316ea784c389eb249b4dd915e0e, role_id=babbae9ac71c4c15817721af5d270a80)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:25.859979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de3a35-d910-49ab-a650-22c4867d9600 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:25.867436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e5ddd-539a-46e8-b1a0-faafac70574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:25.868309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e5ddd-539a-46e8-b1a0-faafac70574b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:25.868674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e5ddd-539a-46e8-b1a0-faafac70574b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:25.868955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e5ddd-539a-46e8-b1a0-faafac70574b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:25.873385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 297/593] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:30:25 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/f4048316ea784c389eb249b4dd915e0e/roles/babbae9ac71c4c15817721af5d270a80 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:30:25.880838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e5ddd-539a-46e8-b1a0-faafac70574b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:25.882344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e5ddd-539a-46e8-b1a0-faafac70574b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:25.886273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de3a35-d910-49ab-a650-22c4867d9600 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-959e5ddd-539a-46e8-b1a0-faafac70574b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:25.901764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:25.902926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:25.903854 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:25.903854 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:25.903854 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:25.908135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 297/594] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:25.909418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e5ddd-539a-46e8-b1a0-faafac70574b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:26.031832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e5ddd-539a-46e8-b1a0-faafac70574b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:26.035672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de3a35-d910-49ab-a650-22c4867d9600 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:26.062347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:26.080906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490a41c-8f96-48b4-ab4a-cc703bece0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:26.081691 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490a41c-8f96-48b4-ab4a-cc703bece0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.082038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490a41c-8f96-48b4-ab4a-cc703bece0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.082408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490a41c-8f96-48b4-ab4a-cc703bece0f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.092150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd62a3-fad8-49f7-bdd7-bdb538050423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:26.092808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd62a3-fad8-49f7-bdd7-bdb538050423 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.093111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd62a3-fad8-49f7-bdd7-bdb538050423 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.093391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd62a3-fad8-49f7-bdd7-bdb538050423 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.094014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd62a3-fad8-49f7-bdd7-bdb538050423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:26.094354 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd62a3-fad8-49f7-bdd7-bdb538050423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:26.097317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd62a3-fad8-49f7-bdd7-bdb538050423 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:26.104624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490a41c-8f96-48b4-ab4a-cc703bece0f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:26.106117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490a41c-8f96-48b4-ab4a-cc703bece0f1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:26.111421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 298/595] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:26.112413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd62a3-fad8-49f7-bdd7-bdb538050423 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c490a41c-8f96-48b4-ab4a-cc703bece0f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:26.125781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:26.127203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:26.128447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:26.128447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:26.128447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:26.131822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 298/596] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:26.133437 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fc73c-f699-409f-a6cf-6de8460814cd None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.133717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fc73c-f699-409f-a6cf-6de8460814cd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.133999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fc73c-f699-409f-a6cf-6de8460814cd None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.140070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 299/597] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:26.140070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fc73c-f699-409f-a6cf-6de8460814cd None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:26.184455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:26.243151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd62a3-fad8-49f7-bdd7-bdb538050423 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:26.260723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fc73c-f699-409f-a6cf-6de8460814cd None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:26.282652 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a104729-aa41-41ce-9263-b966b469aaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:26.283417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a104729-aa41-41ce-9263-b966b469aaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.283801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a104729-aa41-41ce-9263-b966b469aaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.284296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a104729-aa41-41ce-9263-b966b469aaf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.285199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a104729-aa41-41ce-9263-b966b469aaf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:26.285605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a104729-aa41-41ce-9263-b966b469aaf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:26.287198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a104729-aa41-41ce-9263-b966b469aaf1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:26.295491 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e8d6-ba93-4d33-95f9-07da89b41fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:26.296126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e8d6-ba93-4d33-95f9-07da89b41fcb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.296418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e8d6-ba93-4d33-95f9-07da89b41fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.296704 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e8d6-ba93-4d33-95f9-07da89b41fcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.306033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 299/598] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:26.306033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a104729-aa41-41ce-9263-b966b469aaf1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:26.308443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082e8d6-ba93-4d33-95f9-07da89b41fcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:26.309939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082e8d6-ba93-4d33-95f9-07da89b41fcb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-2082e8d6-ba93-4d33-95f9-07da89b41fcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:26.323914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:26.325017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:26.325914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:26.325914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:26.325914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:26.325914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:26.325914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:26.325914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:26.325914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:26.325914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:26.325914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 300/599] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:26.326477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace2b17-4af1-4c41-b768-efeba31ece78 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.326477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace2b17-4af1-4c41-b768-efeba31ece78 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.326720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace2b17-4af1-4c41-b768-efeba31ece78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.345982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2ace2b17-4af1-4c41-b768-efeba31ece78 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:26.347216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2ace2b17-4af1-4c41-b768-efeba31ece78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:26.384582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:26.424480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a104729-aa41-41ce-9263-b966b469aaf1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:26.425076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 301/600] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:26.425944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace2b17-4af1-4c41-b768-efeba31ece78 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:26.450482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:26.476150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b5243-cc2d-4e76-9ca2-fdbf8c627959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:26.476695 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b5243-cc2d-4e76-9ca2-fdbf8c627959 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.477082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b5243-cc2d-4e76-9ca2-fdbf8c627959 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.477474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b5243-cc2d-4e76-9ca2-fdbf8c627959 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.478221 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b5243-cc2d-4e76-9ca2-fdbf8c627959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:26.478627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b5243-cc2d-4e76-9ca2-fdbf8c627959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:26.480704 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b5243-cc2d-4e76-9ca2-fdbf8c627959 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:26.492650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 300/601] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:26.493651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b5243-cc2d-4e76-9ca2-fdbf8c627959 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:26.545354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace2b17-4af1-4c41-b768-efeba31ece78 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:26.587648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5eb27-6cbb-4925-96f9-724eca8b54ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:26.588644 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5eb27-6cbb-4925-96f9-724eca8b54ac None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.589123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5eb27-6cbb-4925-96f9-724eca8b54ac None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.589541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5eb27-6cbb-4925-96f9-724eca8b54ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.597105 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5eb27-6cbb-4925-96f9-724eca8b54ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:26.598826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5eb27-6cbb-4925-96f9-724eca8b54ac None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:26.600447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b5243-cc2d-4e76-9ca2-fdbf8c627959 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-f2e5eb27-6cbb-4925-96f9-724eca8b54ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:26.612219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:26.613314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:26.614369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:26.614369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:26.614369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:26.614369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:26.614369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:26.614369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:26.614369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:26.614369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:26.616545 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 302/602] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:26.617513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5eb27-6cbb-4925-96f9-724eca8b54ac None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:26.643361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d61813-6bdf-4975-b2ad-a75ffc153d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:26.644075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d61813-6bdf-4975-b2ad-a75ffc153d39 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.644458 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d61813-6bdf-4975-b2ad-a75ffc153d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.644985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d61813-6bdf-4975-b2ad-a75ffc153d39 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.645790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d61813-6bdf-4975-b2ad-a75ffc153d39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:26.646201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d61813-6bdf-4975-b2ad-a75ffc153d39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:26.647912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d61813-6bdf-4975-b2ad-a75ffc153d39 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:26.653879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:26.665507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 301/603] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:26.665507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d61813-6bdf-4975-b2ad-a75ffc153d39 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:26.710873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:26.750584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5eb27-6cbb-4925-96f9-724eca8b54ac None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:26.782057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d61813-6bdf-4975-b2ad-a75ffc153d39 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:26.786476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b03b7f-4b59-46e2-b3fd-7dcc02618a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:26.787271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b03b7f-4b59-46e2-b3fd-7dcc02618a80 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.787699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b03b7f-4b59-46e2-b3fd-7dcc02618a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.788154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b03b7f-4b59-46e2-b3fd-7dcc02618a80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.797454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b03b7f-4b59-46e2-b3fd-7dcc02618a80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:26.799267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b03b7f-4b59-46e2-b3fd-7dcc02618a80 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-32b03b7f-4b59-46e2-b3fd-7dcc02618a80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:26.811936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:26.812887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:26.813763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:26.813763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:26.813763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:26.813763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:26.813763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:26.813763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:26.813763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:26.813763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:26.813763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 303/604] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:26.813763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b03b7f-4b59-46e2-b3fd-7dcc02618a80 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:26.826569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faaf8d-54f5-492f-8b34-c245b41f2df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:26.827183 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faaf8d-54f5-492f-8b34-c245b41f2df3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.827479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faaf8d-54f5-492f-8b34-c245b41f2df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.827748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faaf8d-54f5-492f-8b34-c245b41f2df3 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.828438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68faaf8d-54f5-492f-8b34-c245b41f2df3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:26.828778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68faaf8d-54f5-492f-8b34-c245b41f2df3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:26.830203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68faaf8d-54f5-492f-8b34-c245b41f2df3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:26.842179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 302/605] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:26.842930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faaf8d-54f5-492f-8b34-c245b41f2df3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:26.922848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:26.934817 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b03b7f-4b59-46e2-b3fd-7dcc02618a80 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:26.946806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:26.958083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faaf8d-54f5-492f-8b34-c245b41f2df3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:26.972177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fde31-8331-418a-914b-5d1a2e71a973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:26.973004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fde31-8331-418a-914b-5d1a2e71a973 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.973291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fde31-8331-418a-914b-5d1a2e71a973 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.973569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fde31-8331-418a-914b-5d1a2e71a973 None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/e8dec99551ff4c31a17247cde3515acd/roles/babbae9ac71c4c15817721af5d270a80` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:26.991217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fde31-8331-418a-914b-5d1a2e71a973 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=e8dec99551ff4c31a17247cde3515acd, role_id=babbae9ac71c4c15817721af5d270a80)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:26.992806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fde31-8331-418a-914b-5d1a2e71a973 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:26.996077 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59eb739-8f67-4633-a292-27362ba3e7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:26.996743 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eb739-8f67-4633-a292-27362ba3e7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:26.997021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eb739-8f67-4633-a292-27362ba3e7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:26.997298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eb739-8f67-4633-a292-27362ba3e7c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:27.009118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59eb739-8f67-4633-a292-27362ba3e7c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:27.020783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 304/606] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:30:26 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/e8dec99551ff4c31a17247cde3515acd/roles/babbae9ac71c4c15817721af5d270a80 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:30:27.020783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fde31-8331-418a-914b-5d1a2e71a973 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b59eb739-8f67-4633-a292-27362ba3e7c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:27.021078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:27.021939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 303/607] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:27.022791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59eb739-8f67-4633-a292-27362ba3e7c4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:27.149802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59eb739-8f67-4633-a292-27362ba3e7c4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:27.171175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fde31-8331-418a-914b-5d1a2e71a973 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:27.197746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d594d9-1d47-430f-90a1-fcf5991dc67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:27.198472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d594d9-1d47-430f-90a1-fcf5991dc67b None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:27.198754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d594d9-1d47-430f-90a1-fcf5991dc67b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:27.199071 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d594d9-1d47-430f-90a1-fcf5991dc67b None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/30546ba7f5614f878a9ba322c7c55ccb/roles/babbae9ac71c4c15817721af5d270a80` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:27.216447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52e0bc-9ed4-45d5-927c-0430c5dcde34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:27.217285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e0bc-9ed4-45d5-927c-0430c5dcde34 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:27.217565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e0bc-9ed4-45d5-927c-0430c5dcde34 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:27.217837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e0bc-9ed4-45d5-927c-0430c5dcde34 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:27.219548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52e0bc-9ed4-45d5-927c-0430c5dcde34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:27.219861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52e0bc-9ed4-45d5-927c-0430c5dcde34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:27.220865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d594d9-1d47-430f-90a1-fcf5991dc67b None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=30546ba7f5614f878a9ba322c7c55ccb, role_id=babbae9ac71c4c15817721af5d270a80)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:27.222384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d594d9-1d47-430f-90a1-fcf5991dc67b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:27.222767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52e0bc-9ed4-45d5-927c-0430c5dcde34 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:27.238826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 305/608] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:27.239165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:27.244094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 304/609] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:30:27 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/30546ba7f5614f878a9ba322c7c55ccb/roles/babbae9ac71c4c15817721af5d270a80 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:30:27.245836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52e0bc-9ed4-45d5-927c-0430c5dcde34 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:27.250942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d594d9-1d47-430f-90a1-fcf5991dc67b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:27.256885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:27.394010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d594d9-1d47-430f-90a1-fcf5991dc67b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:27.435736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b5da3-3dce-483a-8e1b-174fc8c18d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:27.436426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5da3-3dce-483a-8e1b-174fc8c18d6b None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:27.436744 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5da3-3dce-483a-8e1b-174fc8c18d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:27.437026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5da3-3dce-483a-8e1b-174fc8c18d6b None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/18ec048788904bc58245e878a8644091/roles/babbae9ac71c4c15817721af5d270a80` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:27.458402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b5da3-3dce-483a-8e1b-174fc8c18d6b None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=18ec048788904bc58245e878a8644091, role_id=babbae9ac71c4c15817721af5d270a80)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:27.460651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b5da3-3dce-483a-8e1b-174fc8c18d6b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:27.472762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52e0bc-9ed4-45d5-927c-0430c5dcde34 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:27.478107 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 305/610] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:30:27 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/18ec048788904bc58245e878a8644091/roles/babbae9ac71c4c15817721af5d270a80 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:30:27.537381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eff9a-a116-4c6c-9b5f-1dd9d41b0759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:27.538282 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eff9a-a116-4c6c-9b5f-1dd9d41b0759 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:27.538718 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eff9a-a116-4c6c-9b5f-1dd9d41b0759 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:27.539134 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eff9a-a116-4c6c-9b5f-1dd9d41b0759 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:27.539971 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67eff9a-a116-4c6c-9b5f-1dd9d41b0759 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:27.540447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67eff9a-a116-4c6c-9b5f-1dd9d41b0759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:27.542602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67eff9a-a116-4c6c-9b5f-1dd9d41b0759 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:27.552312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 306/611] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:27.558616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b5da3-3dce-483a-8e1b-174fc8c18d6b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:27.687116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:27.761801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b5da3-3dce-483a-8e1b-174fc8c18d6b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:27.805281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917793b-ecdf-4d33-8fcf-6f7a4e117919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:27.806281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917793b-ecdf-4d33-8fcf-6f7a4e117919 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:27.806764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917793b-ecdf-4d33-8fcf-6f7a4e117919 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:27.807910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917793b-ecdf-4d33-8fcf-6f7a4e117919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:27.822352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917793b-ecdf-4d33-8fcf-6f7a4e117919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:27.824718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917793b-ecdf-4d33-8fcf-6f7a4e117919 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5917793b-ecdf-4d33-8fcf-6f7a4e117919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:27.846855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:27.847873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:27.848780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:27.848780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:27.848780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:27.850578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 306/612] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:27.857345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eff9a-a116-4c6c-9b5f-1dd9d41b0759 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:27.953487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcff6d-56dc-43f9-bffc-56fd94d17211 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:27.953659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcff6d-56dc-43f9-bffc-56fd94d17211 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:27.953932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcff6d-56dc-43f9-bffc-56fd94d17211 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:27.955629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 307/613] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:30:27.959958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:27.961442 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdaf062-bdf1-4502-8e40-f55bfd3adf06 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:27.961940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdaf062-bdf1-4502-8e40-f55bfd3adf06 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:27.962457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdaf062-bdf1-4502-8e40-f55bfd3adf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:27.998096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8fdaf062-bdf1-4502-8e40-f55bfd3adf06 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:28.000393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8fdaf062-bdf1-4502-8e40-f55bfd3adf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:28.037725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eff9a-a116-4c6c-9b5f-1dd9d41b0759 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:28.117775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc64086-2695-44a1-bfb9-38e65ae79e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:28.118397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc64086-2695-44a1-bfb9-38e65ae79e93 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:28.118680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc64086-2695-44a1-bfb9-38e65ae79e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:28.118957 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc64086-2695-44a1-bfb9-38e65ae79e93 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:28.119731 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc64086-2695-44a1-bfb9-38e65ae79e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:28.120221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc64086-2695-44a1-bfb9-38e65ae79e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:28.121643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc64086-2695-44a1-bfb9-38e65ae79e93 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:28.132331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 308/614] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:28.139185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 307/615] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:28.145471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdaf062-bdf1-4502-8e40-f55bfd3adf06 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:28.181246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:28.282702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdaf062-bdf1-4502-8e40-f55bfd3adf06 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:28.346537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fff222-bdcf-484a-b85d-218bc2ed8372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:28.347176 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fff222-bdcf-484a-b85d-218bc2ed8372 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:28.347461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fff222-bdcf-484a-b85d-218bc2ed8372 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:28.347733 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fff222-bdcf-484a-b85d-218bc2ed8372 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:28.365776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fff222-bdcf-484a-b85d-218bc2ed8372 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:28.367536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fff222-bdcf-484a-b85d-218bc2ed8372 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b9fff222-bdcf-484a-b85d-218bc2ed8372 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:28.388951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:28.389942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:28.390877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:28.390877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:28.390877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:28.390877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:28.390877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:28.390877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:28.390877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:28.390877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:28.390877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:28.390877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:28.390877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 309/616] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:28.396089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc64086-2695-44a1-bfb9-38e65ae79e93 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:28.503401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fff222-bdcf-484a-b85d-218bc2ed8372 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:28.547936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc64086-2695-44a1-bfb9-38e65ae79e93 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:28.572605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:28.585215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b416fd9-da13-4fab-bf9b-69a4f0bc1980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:28.586605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b416fd9-da13-4fab-bf9b-69a4f0bc1980 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:28.587050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b416fd9-da13-4fab-bf9b-69a4f0bc1980 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:28.587163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b416fd9-da13-4fab-bf9b-69a4f0bc1980 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:28.587864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b416fd9-da13-4fab-bf9b-69a4f0bc1980 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:28.588880 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b416fd9-da13-4fab-bf9b-69a4f0bc1980 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:28.590668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b416fd9-da13-4fab-bf9b-69a4f0bc1980 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:28.635251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 308/617] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Aug 31 11:30:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:28.637984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b416fd9-da13-4fab-bf9b-69a4f0bc1980 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:28.688855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:28.693161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fff222-bdcf-484a-b85d-218bc2ed8372 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:28.734970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2a80f-01da-4960-a05d-30e5eb65c282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:28.735715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2a80f-01da-4960-a05d-30e5eb65c282 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:28.736053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2a80f-01da-4960-a05d-30e5eb65c282 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:28.736331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2a80f-01da-4960-a05d-30e5eb65c282 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:28.750894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2a80f-01da-4960-a05d-30e5eb65c282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-95f2a80f-01da-4960-a05d-30e5eb65c282 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:28.755284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:28.756217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:28.757122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 310/618] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:28.769473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2a80f-01da-4960-a05d-30e5eb65c282 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:28.823332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b416fd9-da13-4fab-bf9b-69a4f0bc1980 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:28.861121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957ed2e-423c-4b93-8a57-f3c2a6ba4fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:28.862113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957ed2e-423c-4b93-8a57-f3c2a6ba4fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:28.862632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957ed2e-423c-4b93-8a57-f3c2a6ba4fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:28.863108 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957ed2e-423c-4b93-8a57-f3c2a6ba4fe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:28.873805 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957ed2e-423c-4b93-8a57-f3c2a6ba4fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:28.875604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957ed2e-423c-4b93-8a57-f3c2a6ba4fe8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-8957ed2e-423c-4b93-8a57-f3c2a6ba4fe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:28.893559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:28.894727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:28.895624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:28.895624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:28.895624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:28.898023 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 309/619] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:28.904449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957ed2e-423c-4b93-8a57-f3c2a6ba4fe8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:28.978219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:28.996152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2a80f-01da-4960-a05d-30e5eb65c282 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:29.081854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121eb6c8-2ed7-457c-80b4-c712204b07d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:29.082823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb6c8-2ed7-457c-80b4-c712204b07d6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.083305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb6c8-2ed7-457c-80b4-c712204b07d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.083777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb6c8-2ed7-457c-80b4-c712204b07d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.084679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121eb6c8-2ed7-457c-80b4-c712204b07d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:29.085191 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121eb6c8-2ed7-457c-80b4-c712204b07d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:29.086769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121eb6c8-2ed7-457c-80b4-c712204b07d6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:29.097638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957ed2e-423c-4b93-8a57-f3c2a6ba4fe8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:29.101758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 311/620] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:29.107580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121eb6c8-2ed7-457c-80b4-c712204b07d6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:29.141081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:29.172272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625356c8-3cbc-44ee-b973-edc6921d7de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:29.172272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625356c8-3cbc-44ee-b973-edc6921d7de8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.172272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625356c8-3cbc-44ee-b973-edc6921d7de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.172272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625356c8-3cbc-44ee-b973-edc6921d7de8 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.172705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625356c8-3cbc-44ee-b973-edc6921d7de8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:29.172705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625356c8-3cbc-44ee-b973-edc6921d7de8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:29.175291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625356c8-3cbc-44ee-b973-edc6921d7de8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:29.200281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 310/621] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:29.207155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625356c8-3cbc-44ee-b973-edc6921d7de8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:29.286233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121eb6c8-2ed7-457c-80b4-c712204b07d6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:29.324020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0efba9-2fb6-4562-ae2e-be8de1df2974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:29.324718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0efba9-2fb6-4562-ae2e-be8de1df2974 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.325022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0efba9-2fb6-4562-ae2e-be8de1df2974 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.325299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0efba9-2fb6-4562-ae2e-be8de1df2974 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.337426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:29.337678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0efba9-2fb6-4562-ae2e-be8de1df2974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:29.341243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0efba9-2fb6-4562-ae2e-be8de1df2974 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:29.355309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625356c8-3cbc-44ee-b973-edc6921d7de8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-4a0efba9-2fb6-4562-ae2e-be8de1df2974 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:29.359026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:29.359957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:29.360871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:29.360871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:29.360871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:29.360871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 312/622] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:29.361766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c191-437f-4ced-bc82-2a355ff9038d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.362052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c191-437f-4ced-bc82-2a355ff9038d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.362327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c191-437f-4ced-bc82-2a355ff9038d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.367610 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 313/623] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:29.368411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135c191-437f-4ced-bc82-2a355ff9038d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:29.402542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d34815-751f-4f67-9237-fd8de67e6897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:29.403613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d34815-751f-4f67-9237-fd8de67e6897 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.404431 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d34815-751f-4f67-9237-fd8de67e6897 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.405067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d34815-751f-4f67-9237-fd8de67e6897 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.414693 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d34815-751f-4f67-9237-fd8de67e6897 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:29.416573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d34815-751f-4f67-9237-fd8de67e6897 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-75d34815-751f-4f67-9237-fd8de67e6897 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:29.432666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:29.433717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:29.434599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:29.434599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:29.434599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:29.434599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:29.434599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:29.434599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:29.434599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:29.434599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:29.437289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 311/624] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:29.438632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b01877-5da4-4252-89a3-a39e9664c818 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.439047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b01877-5da4-4252-89a3-a39e9664c818 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.439450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b01877-5da4-4252-89a3-a39e9664c818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.451784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:29.462781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e7b01877-5da4-4252-89a3-a39e9664c818 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:29.465520 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e7b01877-5da4-4252-89a3-a39e9664c818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:29.498709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135c191-437f-4ced-bc82-2a355ff9038d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:29.547778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ea003-03d6-4ada-add3-832a140d7bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:29.548752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ea003-03d6-4ada-add3-832a140d7bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.549199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ea003-03d6-4ada-add3-832a140d7bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.549638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ea003-03d6-4ada-add3-832a140d7bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.550482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ea003-03d6-4ada-add3-832a140d7bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:29.550947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ea003-03d6-4ada-add3-832a140d7bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:29.552542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ea003-03d6-4ada-add3-832a140d7bb6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:29.553885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 312/625] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:29.554958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b01877-5da4-4252-89a3-a39e9664c818 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:29.568132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 314/626] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:29.573746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ea003-03d6-4ada-add3-832a140d7bb6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:29.686331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:29.710265 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b01877-5da4-4252-89a3-a39e9664c818 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:29.722969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ea003-03d6-4ada-add3-832a140d7bb6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:29.761189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9a865-691c-442b-85ee-679f3fd0213e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:29.762326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9a865-691c-442b-85ee-679f3fd0213e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.763072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9a865-691c-442b-85ee-679f3fd0213e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.763783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9a865-691c-442b-85ee-679f3fd0213e None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.764881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9a865-691c-442b-85ee-679f3fd0213e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:29.765432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9a865-691c-442b-85ee-679f3fd0213e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:29.767436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9a865-691c-442b-85ee-679f3fd0213e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:29.767608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7d9b0-ae1e-4872-b9d5-1cca7bf3215d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:29.768388 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7d9b0-ae1e-4872-b9d5-1cca7bf3215d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.768786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7d9b0-ae1e-4872-b9d5-1cca7bf3215d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.769131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7d9b0-ae1e-4872-b9d5-1cca7bf3215d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.777678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7d9b0-ae1e-4872-b9d5-1cca7bf3215d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:29.779205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7d9b0-ae1e-4872-b9d5-1cca7bf3215d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:29.784018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 313/627] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:29.785586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2fdb6-77f3-4328-93f9-5bb2c232f705 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.786194 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2fdb6-77f3-4328-93f9-5bb2c232f705 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.786783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2fdb6-77f3-4328-93f9-5bb2c232f705 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.788432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 314/628] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:29.789313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2fdb6-77f3-4328-93f9-5bb2c232f705 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-96a7d9b0-ae1e-4872-b9d5-1cca7bf3215d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:29.796158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:29.797173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:29.798224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:29.798224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:29.798224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:29.798224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:29.798224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:29.798224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:29.798224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:29.798224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:29.798224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:29.798224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:29.798224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 315/629] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Aug 31 11:30:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:29.799983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad1314-b85c-4e70-adf4-f13049a8d48c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.800333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad1314-b85c-4e70-adf4-f13049a8d48c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.800712 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad1314-b85c-4e70-adf4-f13049a8d48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.804827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:29.844516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b5ad1314-b85c-4e70-adf4-f13049a8d48c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:29.844516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b5ad1314-b85c-4e70-adf4-f13049a8d48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:29.924557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2fdb6-77f3-4328-93f9-5bb2c232f705 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:29.932868 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 316/630] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:29.933747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad1314-b85c-4e70-adf4-f13049a8d48c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:29.968774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ad540-b4f0-43eb-a26a-3840c8b61a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:29.969369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ad540-b4f0-43eb-a26a-3840c8b61a77 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:29.969900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ad540-b4f0-43eb-a26a-3840c8b61a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:29.969900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ad540-b4f0-43eb-a26a-3840c8b61a77 None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/3384c24dd081414bbf0a2e3d9f71126b/roles/babbae9ac71c4c15817721af5d270a80` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:29.982763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:29.986625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ad540-b4f0-43eb-a26a-3840c8b61a77 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=3384c24dd081414bbf0a2e3d9f71126b, role_id=babbae9ac71c4c15817721af5d270a80)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:29.988327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ad540-b4f0-43eb-a26a-3840c8b61a77 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:30.003173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 315/631] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:30:29 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/3384c24dd081414bbf0a2e3d9f71126b/roles/babbae9ac71c4c15817721af5d270a80 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:30:30.003861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ad540-b4f0-43eb-a26a-3840c8b61a77 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:30.106380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad1314-b85c-4e70-adf4-f13049a8d48c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:30.154654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:30.186846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da6c7c-dc83-457d-800b-608158ad9640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:30.188662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da6c7c-dc83-457d-800b-608158ad9640 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:30.188948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da6c7c-dc83-457d-800b-608158ad9640 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:30.189299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da6c7c-dc83-457d-800b-608158ad9640 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:30.189961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da6c7c-dc83-457d-800b-608158ad9640 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:30.190439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da6c7c-dc83-457d-800b-608158ad9640 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:30.191957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da6c7c-dc83-457d-800b-608158ad9640 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:30.236472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 317/632] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Aug 31 11:30:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:30.236472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4064d-ba57-4eea-8328-af42d12c79d8 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:30.236472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4064d-ba57-4eea-8328-af42d12c79d8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:30.236472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4064d-ba57-4eea-8328-af42d12c79d8 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:30.236472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 318/633] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:30.236472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4064d-ba57-4eea-8328-af42d12c79d8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:30.255654 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ad540-b4f0-43eb-a26a-3840c8b61a77 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:30.312620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32efcfe-113d-4d7b-896c-c796e49974f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:30.313524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32efcfe-113d-4d7b-896c-c796e49974f6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:30.313815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32efcfe-113d-4d7b-896c-c796e49974f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:30.314130 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32efcfe-113d-4d7b-896c-c796e49974f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:30.329475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32efcfe-113d-4d7b-896c-c796e49974f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b32efcfe-113d-4d7b-896c-c796e49974f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:30.335813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:30.336983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:30.338132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:30.342448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 316/634] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:30.342893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32efcfe-113d-4d7b-896c-c796e49974f6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:30.385997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4064d-ba57-4eea-8328-af42d12c79d8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:30.423214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:30.426151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da62ee-2c73-4b7f-8a77-3f66ed2ab99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:30.426883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da62ee-2c73-4b7f-8a77-3f66ed2ab99a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:30.427166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da62ee-2c73-4b7f-8a77-3f66ed2ab99a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:30.427553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da62ee-2c73-4b7f-8a77-3f66ed2ab99a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:30.440285 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da62ee-2c73-4b7f-8a77-3f66ed2ab99a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-31da62ee-2c73-4b7f-8a77-3f66ed2ab99a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:30.446150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:30.447094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 319/635] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:30.453424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd66298-9f81-4d07-848a-1fb002e94f33 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:30.454180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd66298-9f81-4d07-848a-1fb002e94f33 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:30.454180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd66298-9f81-4d07-848a-1fb002e94f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:30.470212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2dd66298-9f81-4d07-848a-1fb002e94f33 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:30.471408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2dd66298-9f81-4d07-848a-1fb002e94f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:30.485137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32efcfe-113d-4d7b-896c-c796e49974f6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:30.517113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:30.536295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ef7c7-e07c-4c22-b32d-d249082a5edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:30.536965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ef7c7-e07c-4c22-b32d-d249082a5edd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:30.537254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ef7c7-e07c-4c22-b32d-d249082a5edd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:30.537531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ef7c7-e07c-4c22-b32d-d249082a5edd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:30.581939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 320/636] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:30.581939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd66298-9f81-4d07-848a-1fb002e94f33 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:30.693722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd66298-9f81-4d07-848a-1fb002e94f33 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:30.702835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ef7c7-e07c-4c22-b32d-d249082a5edd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:30.704323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ef7c7-e07c-4c22-b32d-d249082a5edd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:30.738448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b54b7-3b18-4aac-95a1-d7114b52a278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:30.739126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b54b7-3b18-4aac-95a1-d7114b52a278 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:30.739404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b54b7-3b18-4aac-95a1-d7114b52a278 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:30.739669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b54b7-3b18-4aac-95a1-d7114b52a278 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:30.740431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b54b7-3b18-4aac-95a1-d7114b52a278 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:30.744105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b54b7-3b18-4aac-95a1-d7114b52a278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:30.745748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b54b7-3b18-4aac-95a1-d7114b52a278 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:30.761068 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 321/637] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:30.761907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b54b7-3b18-4aac-95a1-d7114b52a278 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:30.792958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:30.810584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:30.878178 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 317/638] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:30.879543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ef7c7-e07c-4c22-b32d-d249082a5edd None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:30.929499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b54b7-3b18-4aac-95a1-d7114b52a278 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:30.969751 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bd98e-ce0c-4235-9722-8166f540d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:30.970368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bd98e-ce0c-4235-9722-8166f540d0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:30.970644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bd98e-ce0c-4235-9722-8166f540d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:30.971015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bd98e-ce0c-4235-9722-8166f540d0d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:30.971684 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bd98e-ce0c-4235-9722-8166f540d0d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:30.972089 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bd98e-ce0c-4235-9722-8166f540d0d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:30.973545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bd98e-ce0c-4235-9722-8166f540d0d5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:30.983242 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 322/639] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:30.984183 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bd98e-ce0c-4235-9722-8166f540d0d5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:30.990849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ef7c7-e07c-4c22-b32d-d249082a5edd None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:31.029445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a4663-ae2f-4bc8-a018-8bb7965b4c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:31.030086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a4663-ae2f-4bc8-a018-8bb7965b4c1a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:31.030363 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a4663-ae2f-4bc8-a018-8bb7965b4c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:31.030635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a4663-ae2f-4bc8-a018-8bb7965b4c1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:31.042518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a4663-ae2f-4bc8-a018-8bb7965b4c1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:31.044123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a4663-ae2f-4bc8-a018-8bb7965b4c1a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-408a4663-ae2f-4bc8-a018-8bb7965b4c1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:31.059862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:31.061275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:31.062171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:31.062171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:31.062171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:31.062171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 318/640] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:31.062405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a4663-ae2f-4bc8-a018-8bb7965b4c1a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:31.067120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:31.092448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:31.107660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bd98e-ce0c-4235-9722-8166f540d0d5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:31.153989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d74f41-e801-423b-9f5c-f8ecccabdd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:31.154995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d74f41-e801-423b-9f5c-f8ecccabdd0f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:31.155476 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d74f41-e801-423b-9f5c-f8ecccabdd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:31.155954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d74f41-e801-423b-9f5c-f8ecccabdd0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:31.171513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d74f41-e801-423b-9f5c-f8ecccabdd0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d4d74f41-e801-423b-9f5c-f8ecccabdd0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:31.178660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:31.179697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 323/641] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:31.186126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d74f41-e801-423b-9f5c-f8ecccabdd0f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:31.200411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a4663-ae2f-4bc8-a018-8bb7965b4c1a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:31.243255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5f664-bde7-4d38-97b3-ad677e53700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:31.243953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5f664-bde7-4d38-97b3-ad677e53700a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:31.244263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5f664-bde7-4d38-97b3-ad677e53700a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:31.244697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5f664-bde7-4d38-97b3-ad677e53700a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:31.257053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5f664-bde7-4d38-97b3-ad677e53700a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:31.260353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5f664-bde7-4d38-97b3-ad677e53700a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-28a5f664-bde7-4d38-97b3-ad677e53700a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:31.277940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:31.279026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:31.279921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:31.279921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:31.279921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:31.279921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 319/642] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:31.279921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5f664-bde7-4d38-97b3-ad677e53700a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:31.311099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d74f41-e801-423b-9f5c-f8ecccabdd0f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:31.356617 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0219838-ea69-4067-9abd-a7900f59a43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:31.356825 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:31.357308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219838-ea69-4067-9abd-a7900f59a43f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:31.357602 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219838-ea69-4067-9abd-a7900f59a43f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:31.357870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219838-ea69-4067-9abd-a7900f59a43f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:31.370702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0219838-ea69-4067-9abd-a7900f59a43f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:31.372313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0219838-ea69-4067-9abd-a7900f59a43f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d0219838-ea69-4067-9abd-a7900f59a43f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:31.393150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:31.394880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:31.395904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:31.395904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:31.395904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:31.395904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 324/643] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:31.395904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0219838-ea69-4067-9abd-a7900f59a43f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:31.402160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:31.434733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5f664-bde7-4d38-97b3-ad677e53700a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:31.475776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecf1e0-dc6c-4545-9d2e-18628348222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:31.476450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecf1e0-dc6c-4545-9d2e-18628348222b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:31.476821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecf1e0-dc6c-4545-9d2e-18628348222b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:31.477285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecf1e0-dc6c-4545-9d2e-18628348222b None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:31.479256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecf1e0-dc6c-4545-9d2e-18628348222b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:31.479256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecf1e0-dc6c-4545-9d2e-18628348222b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:31.480824 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecf1e0-dc6c-4545-9d2e-18628348222b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:31.498157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 320/644] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:31.498157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecf1e0-dc6c-4545-9d2e-18628348222b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:31.531573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0219838-ea69-4067-9abd-a7900f59a43f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:31.578682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d0073-ef7e-448a-baff-8e07bc68988f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:31.579659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d0073-ef7e-448a-baff-8e07bc68988f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:31.580133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d0073-ef7e-448a-baff-8e07bc68988f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:31.580641 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d0073-ef7e-448a-baff-8e07bc68988f None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:31.581785 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d0073-ef7e-448a-baff-8e07bc68988f None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:31.582089 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d0073-ef7e-448a-baff-8e07bc68988f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:31.583987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d0073-ef7e-448a-baff-8e07bc68988f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:31.596421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 325/645] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:31.597304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d0073-ef7e-448a-baff-8e07bc68988f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:31.630898 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecf1e0-dc6c-4545-9d2e-18628348222b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:31.655409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:31.684464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ff2d7-1c65-4269-8965-be35c0c7e277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:31.685463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ff2d7-1c65-4269-8965-be35c0c7e277 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:31.686067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ff2d7-1c65-4269-8965-be35c0c7e277 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:31.686721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ff2d7-1c65-4269-8965-be35c0c7e277 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:31.687930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:31.688855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ff2d7-1c65-4269-8965-be35c0c7e277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:31.689926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ff2d7-1c65-4269-8965-be35c0c7e277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:31.692237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ff2d7-1c65-4269-8965-be35c0c7e277 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:31.706953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 321/646] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:31.708632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ff2d7-1c65-4269-8965-be35c0c7e277 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:31.740110 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d0073-ef7e-448a-baff-8e07bc68988f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:31.786359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0fe84-f975-4814-8435-3f52658db4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:31.789119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0fe84-f975-4814-8435-3f52658db4fb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:31.789469 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0fe84-f975-4814-8435-3f52658db4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:31.789750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0fe84-f975-4814-8435-3f52658db4fb None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:31.790948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0fe84-f975-4814-8435-3f52658db4fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:31.791873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0fe84-f975-4814-8435-3f52658db4fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:31.793571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0fe84-f975-4814-8435-3f52658db4fb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:31.807960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 326/647] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:31.809027 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0fe84-f975-4814-8435-3f52658db4fb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:31.850079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ff2d7-1c65-4269-8965-be35c0c7e277 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:31.895336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d3f2-0779-4e55-a945-c1edc2d8a8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:31.895957 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d3f2-0779-4e55-a945-c1edc2d8a8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:31.896232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d3f2-0779-4e55-a945-c1edc2d8a8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:31.896523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d3f2-0779-4e55-a945-c1edc2d8a8d8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:31.907076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874d3f2-0779-4e55-a945-c1edc2d8a8d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:31.908931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874d3f2-0779-4e55-a945-c1edc2d8a8d8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-9874d3f2-0779-4e55-a945-c1edc2d8a8d8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:31.922803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:31.923797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:31.924842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:31.924842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:31.924842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:31.924842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:31.924842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:31.924842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:31.924842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:31.924842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:31.924842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 11:30:31.924842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:31.928694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 322/648] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:31.929560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d3f2-0779-4e55-a945-c1edc2d8a8d8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:31.970654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0fe84-f975-4814-8435-3f52658db4fb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:31.978533 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:32.003818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:32.017549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565d0a8-17b7-463f-be8c-74539d0d7192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:32.018145 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565d0a8-17b7-463f-be8c-74539d0d7192 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.018428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565d0a8-17b7-463f-be8c-74539d0d7192 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.018703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565d0a8-17b7-463f-be8c-74539d0d7192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.031299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565d0a8-17b7-463f-be8c-74539d0d7192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:32.033096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565d0a8-17b7-463f-be8c-74539d0d7192 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8565d0a8-17b7-463f-be8c-74539d0d7192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:32.053528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:32.054708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:32.055740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:32.055740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:32.055740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:32.055740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 327/649] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:32.057510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565d0a8-17b7-463f-be8c-74539d0d7192 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:32.071412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d3f2-0779-4e55-a945-c1edc2d8a8d8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:32.112848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1f2c7-9368-44f2-b6ee-b899e54fe1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:32.113720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1f2c7-9368-44f2-b6ee-b899e54fe1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.114007 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1f2c7-9368-44f2-b6ee-b899e54fe1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.114282 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1f2c7-9368-44f2-b6ee-b899e54fe1c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.122302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1f2c7-9368-44f2-b6ee-b899e54fe1c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:32.124210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1f2c7-9368-44f2-b6ee-b899e54fe1c0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-60f1f2c7-9368-44f2-b6ee-b899e54fe1c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:32.136247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:32.137399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:32.138422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:32.138422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:32.138422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:32.138422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:32.138422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:32.138422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:32.138422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:32.138422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:32.138422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 323/650] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:32.139019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1f2c7-9368-44f2-b6ee-b899e54fe1c0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:32.173359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565d0a8-17b7-463f-be8c-74539d0d7192 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:32.210704 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051dd8f-0ddb-49d9-bfe0-0055084297bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:32.211341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051dd8f-0ddb-49d9-bfe0-0055084297bf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.211618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051dd8f-0ddb-49d9-bfe0-0055084297bf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.211893 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051dd8f-0ddb-49d9-bfe0-0055084297bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.220995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051dd8f-0ddb-49d9-bfe0-0055084297bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:32.222436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051dd8f-0ddb-49d9-bfe0-0055084297bf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6051dd8f-0ddb-49d9-bfe0-0055084297bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:32.234850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:32.235854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:32.236849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:32.236849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:32.236849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:32.236849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:32.236849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:32.236849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:32.236849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:32.236849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:32.236849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 11:30:32.236849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:32.236849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 328/651] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:32.238202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77b52-604f-4072-99b0-1e999f76780f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.238202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77b52-604f-4072-99b0-1e999f76780f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.238202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77b52-604f-4072-99b0-1e999f76780f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.240527 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 329/652] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:32.240527 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a77b52-604f-4072-99b0-1e999f76780f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:32.247316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:32.254593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1f2c7-9368-44f2-b6ee-b899e54fe1c0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:32.281718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:32.293617 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7e5f0-8908-4a95-8c8e-d719d1d1549f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:32.294523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7e5f0-8908-4a95-8c8e-d719d1d1549f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.294813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7e5f0-8908-4a95-8c8e-d719d1d1549f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.295089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7e5f0-8908-4a95-8c8e-d719d1d1549f None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.295835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7e5f0-8908-4a95-8c8e-d719d1d1549f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:32.296150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7e5f0-8908-4a95-8c8e-d719d1d1549f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:32.297635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7e5f0-8908-4a95-8c8e-d719d1d1549f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:32.319311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 324/653] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Aug 31 11:30:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:32.320327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7e5f0-8908-4a95-8c8e-d719d1d1549f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:32.359385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a77b52-604f-4072-99b0-1e999f76780f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:32.401987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e3d00-9ca9-4619-8be2-7453ab8bc911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:32.402574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e3d00-9ca9-4619-8be2-7453ab8bc911 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.402790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e3d00-9ca9-4619-8be2-7453ab8bc911 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.403100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e3d00-9ca9-4619-8be2-7453ab8bc911 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.403747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e3d00-9ca9-4619-8be2-7453ab8bc911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:32.404070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e3d00-9ca9-4619-8be2-7453ab8bc911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:32.405488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e3d00-9ca9-4619-8be2-7453ab8bc911 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:32.417774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 330/654] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:32.423004 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e3d00-9ca9-4619-8be2-7453ab8bc911 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:32.438107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7e5f0-8908-4a95-8c8e-d719d1d1549f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:32.475379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0c768-3c16-46a6-b889-2592ad55c182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:32.476328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0c768-3c16-46a6-b889-2592ad55c182 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.476847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0c768-3c16-46a6-b889-2592ad55c182 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.477301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0c768-3c16-46a6-b889-2592ad55c182 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.478182 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f0c768-3c16-46a6-b889-2592ad55c182 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:32.479365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f0c768-3c16-46a6-b889-2592ad55c182 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:32.480850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f0c768-3c16-46a6-b889-2592ad55c182 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:32.499805 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 325/655] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:32.499805 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad1a7f-211d-40b5-a428-797de2eba3c0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.499805 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad1a7f-211d-40b5-a428-797de2eba3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.499805 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad1a7f-211d-40b5-a428-797de2eba3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.518797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cdad1a7f-211d-40b5-a428-797de2eba3c0 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:32.519994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cdad1a7f-211d-40b5-a428-797de2eba3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:32.531020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:32.535133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:32.542924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e3d00-9ca9-4619-8be2-7453ab8bc911 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:32.580216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1310f-ff75-42a6-8fd1-85db901544d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:32.580916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1310f-ff75-42a6-8fd1-85db901544d6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.581265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1310f-ff75-42a6-8fd1-85db901544d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.581547 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1310f-ff75-42a6-8fd1-85db901544d6 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.582303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1310f-ff75-42a6-8fd1-85db901544d6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:32.582597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1310f-ff75-42a6-8fd1-85db901544d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:32.584029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1310f-ff75-42a6-8fd1-85db901544d6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:32.598960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 326/656] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:32.599816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad1a7f-211d-40b5-a428-797de2eba3c0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:32.605350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 331/657] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Aug 31 11:30:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:32.606144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1310f-ff75-42a6-8fd1-85db901544d6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:32.726704 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad1a7f-211d-40b5-a428-797de2eba3c0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:32.727481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1310f-ff75-42a6-8fd1-85db901544d6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:32.771160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1a52c-e39b-4426-be0a-4525ce5fe9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:32.771627 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a8b73-4791-406f-84c7-8ca5dc202572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:32.772240 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a8b73-4791-406f-84c7-8ca5dc202572 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.772551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a8b73-4791-406f-84c7-8ca5dc202572 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.772844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a8b73-4791-406f-84c7-8ca5dc202572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.773673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1a52c-e39b-4426-be0a-4525ce5fe9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.774014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1a52c-e39b-4426-be0a-4525ce5fe9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.774296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1a52c-e39b-4426-be0a-4525ce5fe9a4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.783992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1a52c-e39b-4426-be0a-4525ce5fe9a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:32.785530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1a52c-e39b-4426-be0a-4525ce5fe9a4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:32.794379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-52b1a52c-e39b-4426-be0a-4525ce5fe9a4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:32.802188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:32.803152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:32.804096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:32.804096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:32.804096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:32.804096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:32.804096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:32.804096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:32.804096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:32.804096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:32.804096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 11:30:32.804096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:32.804096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 332/658] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:32.804740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1a52c-e39b-4426-be0a-4525ce5fe9a4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:32.825215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:32.878481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a8b73-4791-406f-84c7-8ca5dc202572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:32.879977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a8b73-4791-406f-84c7-8ca5dc202572 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:32.925099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1a52c-e39b-4426-be0a-4525ce5fe9a4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:32.966142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb10d7-e26f-4425-88fe-4fb2997f6b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:32.966681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb10d7-e26f-4425-88fe-4fb2997f6b47 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:32.966962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb10d7-e26f-4425-88fe-4fb2997f6b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:32.967229 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb10d7-e26f-4425-88fe-4fb2997f6b47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:32.967945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb10d7-e26f-4425-88fe-4fb2997f6b47 None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=f4048316ea784c389eb249b4dd915e0e) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:32.968251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb10d7-e26f-4425-88fe-4fb2997f6b47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:32.969727 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb10d7-e26f-4425-88fe-4fb2997f6b47 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:32.976088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 333/659] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:32 2022] GET /identity/v3/role_assignments?user.id=f4048316ea784c389eb249b4dd915e0e&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:32.977447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb10d7-e26f-4425-88fe-4fb2997f6b47 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:32.991534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 327/660] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:32.992853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a8b73-4791-406f-84c7-8ca5dc202572 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:33.062825 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:33.102089 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:33.125667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb10d7-e26f-4425-88fe-4fb2997f6b47 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:33.140530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a8b73-4791-406f-84c7-8ca5dc202572 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:33.195507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965085f-2eb7-4413-8f33-8793c2c9114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:33.195507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965085f-2eb7-4413-8f33-8793c2c9114e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:33.195507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965085f-2eb7-4413-8f33-8793c2c9114e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:33.195507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965085f-2eb7-4413-8f33-8793c2c9114e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:33.201416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b85b08-bac4-4c57-ba37-5855458867d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:33.203351 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b85b08-bac4-4c57-ba37-5855458867d1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:33.203351 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b85b08-bac4-4c57-ba37-5855458867d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:33.203351 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b85b08-bac4-4c57-ba37-5855458867d1 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:33.203620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b85b08-bac4-4c57-ba37-5855458867d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:33.204693 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b85b08-bac4-4c57-ba37-5855458867d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:33.206966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b85b08-bac4-4c57-ba37-5855458867d1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:33.231287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 328/661] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Aug 31 11:30:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:33.232191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b85b08-bac4-4c57-ba37-5855458867d1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:33.294675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965085f-2eb7-4413-8f33-8793c2c9114e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:33.296289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965085f-2eb7-4413-8f33-8793c2c9114e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:33.363739 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b85b08-bac4-4c57-ba37-5855458867d1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:33.366128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:33.395768 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:33.409001 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254a9ba-18b0-48c5-a83f-693eae761b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:33.409959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254a9ba-18b0-48c5-a83f-693eae761b6a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:33.410295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254a9ba-18b0-48c5-a83f-693eae761b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:33.410560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254a9ba-18b0-48c5-a83f-693eae761b6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:33.426155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254a9ba-18b0-48c5-a83f-693eae761b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:33.430835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 334/662] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:33.430835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965085f-2eb7-4413-8f33-8793c2c9114e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-e254a9ba-18b0-48c5-a83f-693eae761b6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:33.435916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:33.439348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 329/663] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:33.440222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254a9ba-18b0-48c5-a83f-693eae761b6a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:33.554198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965085f-2eb7-4413-8f33-8793c2c9114e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:33.568564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254a9ba-18b0-48c5-a83f-693eae761b6a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:33.605242 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da8595-39df-4305-a7a8-72619bf218b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:33.605970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da8595-39df-4305-a7a8-72619bf218b3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:33.606356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da8595-39df-4305-a7a8-72619bf218b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:33.606651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da8595-39df-4305-a7a8-72619bf218b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:33.618769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da8595-39df-4305-a7a8-72619bf218b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:33.620298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da8595-39df-4305-a7a8-72619bf218b3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:33.623000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e31d5-45b8-4abc-b5a6-069f595429e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:33.623000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e31d5-45b8-4abc-b5a6-069f595429e3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:33.623000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e31d5-45b8-4abc-b5a6-069f595429e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:33.623000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e31d5-45b8-4abc-b5a6-069f595429e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-82da8595-39df-4305-a7a8-72619bf218b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:33.644212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:33.645238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:33.646129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:33.646129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:33.646129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:33.646310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 335/664] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:33.647419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da8595-39df-4305-a7a8-72619bf218b3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:33.662950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:33.694840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:33.715249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e31d5-45b8-4abc-b5a6-069f595429e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:33.716773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e31d5-45b8-4abc-b5a6-069f595429e3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:33.780917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da8595-39df-4305-a7a8-72619bf218b3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:33.831578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05dd61-abc7-436c-99b5-ce74ea3cd55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:33.833254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05dd61-abc7-436c-99b5-ce74ea3cd55a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:33.833540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05dd61-abc7-436c-99b5-ce74ea3cd55a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:33.833820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05dd61-abc7-436c-99b5-ce74ea3cd55a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:33.850530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05dd61-abc7-436c-99b5-ce74ea3cd55a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:33.851992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05dd61-abc7-436c-99b5-ce74ea3cd55a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:33.862391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 330/665] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:33.863238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e31d5-45b8-4abc-b5a6-069f595429e3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6f05dd61-abc7-436c-99b5-ce74ea3cd55a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:33.875778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:33.877098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:33.878130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:33.878130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:33.878130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:33.882301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 336/666] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:33.882301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05dd61-abc7-436c-99b5-ce74ea3cd55a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:33.948732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:33.983552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:34.004797 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05dd61-abc7-436c-99b5-ce74ea3cd55a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:34.005202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e31d5-45b8-4abc-b5a6-069f595429e3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:34.054840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c95ee-ab3d-4774-b6c3-ad90b58ff299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:34.055464 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c95ee-ab3d-4774-b6c3-ad90b58ff299 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:34.055974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c95ee-ab3d-4774-b6c3-ad90b58ff299 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:34.056256 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c95ee-ab3d-4774-b6c3-ad90b58ff299 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:34.057697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c95ee-ab3d-4774-b6c3-ad90b58ff299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:34.058042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c95ee-ab3d-4774-b6c3-ad90b58ff299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:34.059422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c95ee-ab3d-4774-b6c3-ad90b58ff299 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:34.060005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913e9a4-c763-4b24-a532-4e19f1769bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:34.060713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913e9a4-c763-4b24-a532-4e19f1769bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:34.061084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913e9a4-c763-4b24-a532-4e19f1769bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:34.061425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913e9a4-c763-4b24-a532-4e19f1769bb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:34.062195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913e9a4-c763-4b24-a532-4e19f1769bb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:34.062575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913e9a4-c763-4b24-a532-4e19f1769bb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:34.064111 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913e9a4-c763-4b24-a532-4e19f1769bb6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:34.079481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 331/667] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:34.079481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913e9a4-c763-4b24-a532-4e19f1769bb6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:34.079998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 337/668] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:34.081116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c95ee-ab3d-4774-b6c3-ad90b58ff299 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:34.205513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c95ee-ab3d-4774-b6c3-ad90b58ff299 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:34.207845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913e9a4-c763-4b24-a532-4e19f1769bb6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:34.233476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:34.245743 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169eed05-25f6-47c4-a350-812dbea4dd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:34.246351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169eed05-25f6-47c4-a350-812dbea4dd44 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:34.246664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169eed05-25f6-47c4-a350-812dbea4dd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:34.246899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169eed05-25f6-47c4-a350-812dbea4dd44 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:34.247540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169eed05-25f6-47c4-a350-812dbea4dd44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:34.247843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169eed05-25f6-47c4-a350-812dbea4dd44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:34.249226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169eed05-25f6-47c4-a350-812dbea4dd44 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:34.252163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b4daa-7ce0-4135-954c-34c26be08665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:34.252847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b4daa-7ce0-4135-954c-34c26be08665 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:34.253129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b4daa-7ce0-4135-954c-34c26be08665 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:34.253405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b4daa-7ce0-4135-954c-34c26be08665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:34.267289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 338/669] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:34.267289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169eed05-25f6-47c4-a350-812dbea4dd44 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:34.267530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b4daa-7ce0-4135-954c-34c26be08665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:34.268185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b4daa-7ce0-4135-954c-34c26be08665 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:34.269857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-da2b4daa-7ce0-4135-954c-34c26be08665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:34.288966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:34.289934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:34.290849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:34.290849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:34.290849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:34.296005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 332/670] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:34.297010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b4daa-7ce0-4135-954c-34c26be08665 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:34.410765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169eed05-25f6-47c4-a350-812dbea4dd44 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:34.423001 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b4daa-7ce0-4135-954c-34c26be08665 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:34.459468 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426debc-53ea-402d-b71f-24f5960bdf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:34.460822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426debc-53ea-402d-b71f-24f5960bdf1a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:34.461109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426debc-53ea-402d-b71f-24f5960bdf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:34.461643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426debc-53ea-402d-b71f-24f5960bdf1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:34.469586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0ce12-7198-484a-b3d9-16afc5de3fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:34.472139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0ce12-7198-484a-b3d9-16afc5de3fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:34.472438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0ce12-7198-484a-b3d9-16afc5de3fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:34.472765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0ce12-7198-484a-b3d9-16afc5de3fc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:34.473323 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426debc-53ea-402d-b71f-24f5960bdf1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:34.474921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426debc-53ea-402d-b71f-24f5960bdf1a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:34.485848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0ce12-7198-484a-b3d9-16afc5de3fc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:34.487635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0ce12-7198-484a-b3d9-16afc5de3fc1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f426debc-53ea-402d-b71f-24f5960bdf1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:34.492864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:34.493773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:34.495259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:34.495259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:34.495259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:34.495259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:34.495259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:34.495259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:34.495259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:34.495259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:34.495259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 339/671] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:34.495885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426debc-53ea-402d-b71f-24f5960bdf1a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d0d0ce12-7198-484a-b3d9-16afc5de3fc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:34.507695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:34.508748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:34.509659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:34.509659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:34.509659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:34.510489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 333/672] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:34.511209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0ce12-7198-484a-b3d9-16afc5de3fc1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:34.548846 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:34.583459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:34.634703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0ce12-7198-484a-b3d9-16afc5de3fc1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:34.638351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426debc-53ea-402d-b71f-24f5960bdf1a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:34.675728 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cce338-0ff0-4418-937c-7f2d8d03fcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:34.679334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca257092-9577-4a44-8290-333e50ae3811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:34.679501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cce338-0ff0-4418-937c-7f2d8d03fcfe None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:34.679620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca257092-9577-4a44-8290-333e50ae3811 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:34.679711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cce338-0ff0-4418-937c-7f2d8d03fcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:34.679806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca257092-9577-4a44-8290-333e50ae3811 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:34.679903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cce338-0ff0-4418-937c-7f2d8d03fcfe None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:34.679995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca257092-9577-4a44-8290-333e50ae3811 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:34.680086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cce338-0ff0-4418-937c-7f2d8d03fcfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:34.680086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cce338-0ff0-4418-937c-7f2d8d03fcfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:34.680086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cce338-0ff0-4418-937c-7f2d8d03fcfe None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:34.687303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca257092-9577-4a44-8290-333e50ae3811 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:34.688996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca257092-9577-4a44-8290-333e50ae3811 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ca257092-9577-4a44-8290-333e50ae3811 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:34.702157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:34.703210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:34.704295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:34.704295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:34.704295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:34.704295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:34.704295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:34.704295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:34.704295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:34.704295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:34.704988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 334/673] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:34.705323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cce338-0ff0-4418-937c-7f2d8d03fcfe None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:34.709778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 340/674] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:34.710946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca257092-9577-4a44-8290-333e50ae3811 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:34.828777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:34.848138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca257092-9577-4a44-8290-333e50ae3811 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:34.849025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cce338-0ff0-4418-937c-7f2d8d03fcfe None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:34.877669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:34.891959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790e2c9-b333-48fe-bebf-1472641ec099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:34.893154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d24fcf-8ee5-4ace-99ed-e344f0033037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:34.893308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790e2c9-b333-48fe-bebf-1472641ec099 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:34.893720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790e2c9-b333-48fe-bebf-1472641ec099 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:34.896156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d24fcf-8ee5-4ace-99ed-e344f0033037 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:34.896156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d24fcf-8ee5-4ace-99ed-e344f0033037 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:34.896156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d24fcf-8ee5-4ace-99ed-e344f0033037 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:34.896156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d24fcf-8ee5-4ace-99ed-e344f0033037 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:34.896156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d24fcf-8ee5-4ace-99ed-e344f0033037 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:34.896921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790e2c9-b333-48fe-bebf-1472641ec099 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:34.897575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d24fcf-8ee5-4ace-99ed-e344f0033037 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:34.897956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790e2c9-b333-48fe-bebf-1472641ec099 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:34.898498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790e2c9-b333-48fe-bebf-1472641ec099 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:34.900177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790e2c9-b333-48fe-bebf-1472641ec099 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:34.917898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 341/675] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:34.917898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d24fcf-8ee5-4ace-99ed-e344f0033037 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:34.926243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 335/676] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:34.926243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790e2c9-b333-48fe-bebf-1472641ec099 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:35.042897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d24fcf-8ee5-4ace-99ed-e344f0033037 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:35.057044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790e2c9-b333-48fe-bebf-1472641ec099 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:35.081916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e83ed-9a51-44e1-8f09-05293b379658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:35.082866 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e83ed-9a51-44e1-8f09-05293b379658 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:35.084611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e83ed-9a51-44e1-8f09-05293b379658 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:35.084611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e83ed-9a51-44e1-8f09-05293b379658 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:35.092522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e83ed-9a51-44e1-8f09-05293b379658 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:35.093907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723c4d8-9fb8-4e3b-a291-2cb369d990e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:35.094074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e83ed-9a51-44e1-8f09-05293b379658 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:35.094818 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723c4d8-9fb8-4e3b-a291-2cb369d990e2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:35.095234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723c4d8-9fb8-4e3b-a291-2cb369d990e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:35.095359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723c4d8-9fb8-4e3b-a291-2cb369d990e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:35.096054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723c4d8-9fb8-4e3b-a291-2cb369d990e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:35.096366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723c4d8-9fb8-4e3b-a291-2cb369d990e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:35.097842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723c4d8-9fb8-4e3b-a291-2cb369d990e2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-775e83ed-9a51-44e1-8f09-05293b379658 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:35.105997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:35.106898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:35.107816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:35.107816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:35.107816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:35.107816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:35.107816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:35.107816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:35.107816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:35.107816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:35.107816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 342/677] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:35.108311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e83ed-9a51-44e1-8f09-05293b379658 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:35.119909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 336/678] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:35.119909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723c4d8-9fb8-4e3b-a291-2cb369d990e2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:35.131088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:35.155709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:35.222742 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e83ed-9a51-44e1-8f09-05293b379658 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:35.233052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723c4d8-9fb8-4e3b-a291-2cb369d990e2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:35.258723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081a82c-3092-4bc6-ab83-195797ce6833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:35.259361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081a82c-3092-4bc6-ab83-195797ce6833 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:35.259647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081a82c-3092-4bc6-ab83-195797ce6833 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:35.260688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081a82c-3092-4bc6-ab83-195797ce6833 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:35.268736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081a82c-3092-4bc6-ab83-195797ce6833 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:35.270170 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4980d98-9d15-4bfe-aa70-11c91d00a4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:35.270336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081a82c-3092-4bc6-ab83-195797ce6833 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:35.270776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980d98-9d15-4bfe-aa70-11c91d00a4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:35.271057 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980d98-9d15-4bfe-aa70-11c91d00a4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:35.271333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980d98-9d15-4bfe-aa70-11c91d00a4d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:35.272064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4980d98-9d15-4bfe-aa70-11c91d00a4d5 None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=e8dec99551ff4c31a17247cde3515acd) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:35.272376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4980d98-9d15-4bfe-aa70-11c91d00a4d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:35.273832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4980d98-9d15-4bfe-aa70-11c91d00a4d5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:35.280585 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 337/679] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:35 2022] GET /identity/v3/role_assignments?user.id=e8dec99551ff4c31a17247cde3515acd&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5081a82c-3092-4bc6-ab83-195797ce6833 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:35.282268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:35.283186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:35.284207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:35.284207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:35.284207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:35.284207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:35.284207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:35.284207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:35.284207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:35.284207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:35.284207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:35.284207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:35.285244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f3d74-1fb9-4ed3-be9b-ec8f89566ffd None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:35.285244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f3d74-1fb9-4ed3-be9b-ec8f89566ffd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:35.285244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f3d74-1fb9-4ed3-be9b-ec8f89566ffd None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:35.288369 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 343/680] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:35.289420 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081a82c-3092-4bc6-ab83-195797ce6833 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:35.291419 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 338/681] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:35.292393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f3d74-1fb9-4ed3-be9b-ec8f89566ffd None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:35.407471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:35.428253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081a82c-3092-4bc6-ab83-195797ce6833 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:35.430389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:35.451886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f3d74-1fb9-4ed3-be9b-ec8f89566ffd None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:35.484746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c0af2-eb10-4302-8496-a4e6d5639a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:35.485428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c0af2-eb10-4302-8496-a4e6d5639a10 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:35.485632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c0af2-eb10-4302-8496-a4e6d5639a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:35.485904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c0af2-eb10-4302-8496-a4e6d5639a10 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:35.486597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c0af2-eb10-4302-8496-a4e6d5639a10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:35.486901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c0af2-eb10-4302-8496-a4e6d5639a10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:35.488701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c0af2-eb10-4302-8496-a4e6d5639a10 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:35.501708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef550e9-5637-4df4-a2ff-50658ae8be10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:35.502325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef550e9-5637-4df4-a2ff-50658ae8be10 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:35.502610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef550e9-5637-4df4-a2ff-50658ae8be10 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:35.503056 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef550e9-5637-4df4-a2ff-50658ae8be10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:35.503801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef550e9-5637-4df4-a2ff-50658ae8be10 None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=30546ba7f5614f878a9ba322c7c55ccb) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:35.504303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef550e9-5637-4df4-a2ff-50658ae8be10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:35.504716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 344/682] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:35.505439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c0af2-eb10-4302-8496-a4e6d5639a10 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:35.505935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef550e9-5637-4df4-a2ff-50658ae8be10 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:35.517153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 339/683] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:35 2022] GET /identity/v3/role_assignments?user.id=30546ba7f5614f878a9ba322c7c55ccb&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:35.519398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6a4eb-fb61-4059-924a-462cc3a4c15b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:35.519905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6a4eb-fb61-4059-924a-462cc3a4c15b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:35.520975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6a4eb-fb61-4059-924a-462cc3a4c15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:35.552333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-81d6a4eb-fb61-4059-924a-462cc3a4c15b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:35.553977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-81d6a4eb-fb61-4059-924a-462cc3a4c15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:35.653340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c0af2-eb10-4302-8496-a4e6d5639a10 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:35.660304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 340/684] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:35.661617 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6a4eb-fb61-4059-924a-462cc3a4c15b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:35.741806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e498bbb-505e-4d82-8319-4bbcb5ad1f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:35.742606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498bbb-505e-4d82-8319-4bbcb5ad1f52 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:35.742887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498bbb-505e-4d82-8319-4bbcb5ad1f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:35.743159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498bbb-505e-4d82-8319-4bbcb5ad1f52 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:35.743905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e498bbb-505e-4d82-8319-4bbcb5ad1f52 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:35.744860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e498bbb-505e-4d82-8319-4bbcb5ad1f52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:35.746365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e498bbb-505e-4d82-8319-4bbcb5ad1f52 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:35.787760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:35.799546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 345/685] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Aug 31 11:30:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:35.802155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e498bbb-505e-4d82-8319-4bbcb5ad1f52 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:35.841628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6a4eb-fb61-4059-924a-462cc3a4c15b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:35.856945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:35.934521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390aab0-6316-4783-8b70-a1ab288b2b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:35.935803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390aab0-6316-4783-8b70-a1ab288b2b5f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:35.936312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390aab0-6316-4783-8b70-a1ab288b2b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:35.940900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390aab0-6316-4783-8b70-a1ab288b2b5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:35.942161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390aab0-6316-4783-8b70-a1ab288b2b5f None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=18ec048788904bc58245e878a8644091) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:35.942700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390aab0-6316-4783-8b70-a1ab288b2b5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:35.944483 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390aab0-6316-4783-8b70-a1ab288b2b5f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:35.965069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 341/686] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:35 2022] GET /identity/v3/role_assignments?user.id=18ec048788904bc58245e878a8644091&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:35.989642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e498bbb-505e-4d82-8319-4bbcb5ad1f52 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:36.033495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946d129-540e-455b-ac77-06866942ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:36.033706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d129-540e-455b-ac77-06866942ff1c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:36.034760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d129-540e-455b-ac77-06866942ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:36.034760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d129-540e-455b-ac77-06866942ff1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:36.048648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390aab0-6316-4783-8b70-a1ab288b2b5f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:36.121932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946d129-540e-455b-ac77-06866942ff1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:36.123419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946d129-540e-455b-ac77-06866942ff1c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:36.177198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:36.270534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390aab0-6316-4783-8b70-a1ab288b2b5f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:36.318121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:36.332298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60217f7a-cd08-485c-b0d0-a6f9b2779a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:36.333307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60217f7a-cd08-485c-b0d0-a6f9b2779a36 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:36.333808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60217f7a-cd08-485c-b0d0-a6f9b2779a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:36.334279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60217f7a-cd08-485c-b0d0-a6f9b2779a36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:36.338303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 346/687] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 11:30:36.347138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946d129-540e-455b-ac77-06866942ff1c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:36.363322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60217f7a-cd08-485c-b0d0-a6f9b2779a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-60217f7a-cd08-485c-b0d0-a6f9b2779a36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:36.368407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:36.370283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:36.371373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 342/688] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:36.379180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60217f7a-cd08-485c-b0d0-a6f9b2779a36 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:36.500234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60217f7a-cd08-485c-b0d0-a6f9b2779a36 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:36.547241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cc8f9-1c1d-4c9c-93a0-a8cdd6105652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:36.547788 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cc8f9-1c1d-4c9c-93a0-a8cdd6105652 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:36.548072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cc8f9-1c1d-4c9c-93a0-a8cdd6105652 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:36.548349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cc8f9-1c1d-4c9c-93a0-a8cdd6105652 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:36.549050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048cc8f9-1c1d-4c9c-93a0-a8cdd6105652 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:36.549359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048cc8f9-1c1d-4c9c-93a0-a8cdd6105652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:36.556690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048cc8f9-1c1d-4c9c-93a0-a8cdd6105652 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:36.562910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 343/689] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:36.574011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cc8f9-1c1d-4c9c-93a0-a8cdd6105652 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:36.584292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946d129-540e-455b-ac77-06866942ff1c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:36.620830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:36.657194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ea2d7-1bb1-4045-8988-62729d6fca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:36.657831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ea2d7-1bb1-4045-8988-62729d6fca48 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:36.658108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ea2d7-1bb1-4045-8988-62729d6fca48 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:36.658375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ea2d7-1bb1-4045-8988-62729d6fca48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:36.697568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ea2d7-1bb1-4045-8988-62729d6fca48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:36.698980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ea2d7-1bb1-4045-8988-62729d6fca48 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:36.700272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d06ea2d7-1bb1-4045-8988-62729d6fca48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:36.743582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:36.744682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:36.745588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:36.745588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:36.745588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:36.749787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 347/690] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:36.757790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ea2d7-1bb1-4045-8988-62729d6fca48 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:36.769686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cc8f9-1c1d-4c9c-93a0-a8cdd6105652 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:36.837462 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503a2fb-9c89-4cf7-92cc-aa09620b3516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:36.839086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a2fb-9c89-4cf7-92cc-aa09620b3516 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:36.839543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a2fb-9c89-4cf7-92cc-aa09620b3516 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:36.840018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a2fb-9c89-4cf7-92cc-aa09620b3516 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:36.871797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503a2fb-9c89-4cf7-92cc-aa09620b3516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:36.882231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503a2fb-9c89-4cf7-92cc-aa09620b3516 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-3503a2fb-9c89-4cf7-92cc-aa09620b3516 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:36.922047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:36.923052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:36.923991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:36.923991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:36.923991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:36.931427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 344/691] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:36.946127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503a2fb-9c89-4cf7-92cc-aa09620b3516 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:37.026987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ea2d7-1bb1-4045-8988-62729d6fca48 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:37.111015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b14a1-cbca-4753-af10-5056a5e9ec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:37.111678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b14a1-cbca-4753-af10-5056a5e9ec86 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:37.112027 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b14a1-cbca-4753-af10-5056a5e9ec86 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:37.112366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b14a1-cbca-4753-af10-5056a5e9ec86 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:37.113126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b14a1-cbca-4753-af10-5056a5e9ec86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:37.113513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b14a1-cbca-4753-af10-5056a5e9ec86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:37.114960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b14a1-cbca-4753-af10-5056a5e9ec86 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:37.168206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 348/692] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:37.176814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b14a1-cbca-4753-af10-5056a5e9ec86 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:37.187440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503a2fb-9c89-4cf7-92cc-aa09620b3516 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:37.198720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:37.301215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:37.301893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7804aa-2121-4b8a-a62f-ce0f089e5251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:37.302291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7804aa-2121-4b8a-a62f-ce0f089e5251 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:37.308513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7804aa-2121-4b8a-a62f-ce0f089e5251 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:37.308513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7804aa-2121-4b8a-a62f-ce0f089e5251 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:37.308513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7804aa-2121-4b8a-a62f-ce0f089e5251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:37.308513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7804aa-2121-4b8a-a62f-ce0f089e5251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:37.313310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7804aa-2121-4b8a-a62f-ce0f089e5251 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:37.368356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 345/693] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:37.378161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7804aa-2121-4b8a-a62f-ce0f089e5251 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:37.564233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b14a1-cbca-4753-af10-5056a5e9ec86 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:37.598268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7804aa-2121-4b8a-a62f-ce0f089e5251 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:37.647690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da739653-cca4-4f1c-bc71-e0464bd72f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:37.648279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da739653-cca4-4f1c-bc71-e0464bd72f7f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:37.648601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da739653-cca4-4f1c-bc71-e0464bd72f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:37.652848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da739653-cca4-4f1c-bc71-e0464bd72f7f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:37.673542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da739653-cca4-4f1c-bc71-e0464bd72f7f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:37.675040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da739653-cca4-4f1c-bc71-e0464bd72f7f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-da739653-cca4-4f1c-bc71-e0464bd72f7f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:37.693032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:37.694048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:37.695005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:37.695005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:37.695005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:37.695005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:37.695005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:37.695005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:37.695005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:37.695005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:37.695005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:37.695005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:37.695005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 346/694] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Aug 31 11:30:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:37.705451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da739653-cca4-4f1c-bc71-e0464bd72f7f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:37.709214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3570c9d-753c-427e-bf7f-be2073fa91ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:37.709867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3570c9d-753c-427e-bf7f-be2073fa91ec None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:37.710151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3570c9d-753c-427e-bf7f-be2073fa91ec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:37.710453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3570c9d-753c-427e-bf7f-be2073fa91ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:37.727543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3570c9d-753c-427e-bf7f-be2073fa91ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:37.731035 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3570c9d-753c-427e-bf7f-be2073fa91ec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d3570c9d-753c-427e-bf7f-be2073fa91ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:37.755178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:37.756135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:37.757096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:37.757096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:37.757096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:37.757096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:37.757096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:37.757096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:37.757096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:37.757096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:37.765289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 349/695] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 589 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:37.766204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3570c9d-753c-427e-bf7f-be2073fa91ec None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:37.787019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:37.929177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da739653-cca4-4f1c-bc71-e0464bd72f7f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:37.946362 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:37.963677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3570c9d-753c-427e-bf7f-be2073fa91ec None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:38.003594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987fdfd-7395-46c9-84e5-e0ee23283425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:38.005492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987fdfd-7395-46c9-84e5-e0ee23283425 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.008523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987fdfd-7395-46c9-84e5-e0ee23283425 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.008523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987fdfd-7395-46c9-84e5-e0ee23283425 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.008523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987fdfd-7395-46c9-84e5-e0ee23283425 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:38.008523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987fdfd-7395-46c9-84e5-e0ee23283425 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:38.010406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987fdfd-7395-46c9-84e5-e0ee23283425 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:38.035116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 347/696] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Aug 31 11:30:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:38.035964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640545ac-c218-4951-921d-dc49b004b4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:38.036745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640545ac-c218-4951-921d-dc49b004b4dc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.037032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640545ac-c218-4951-921d-dc49b004b4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.037305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640545ac-c218-4951-921d-dc49b004b4dc None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.038006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640545ac-c218-4951-921d-dc49b004b4dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:38.038312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640545ac-c218-4951-921d-dc49b004b4dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:38.040808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886c3c1-202c-437e-ad75-d1b052508dc7 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.041174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886c3c1-202c-437e-ad75-d1b052508dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.042143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886c3c1-202c-437e-ad75-d1b052508dc7 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.043908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640545ac-c218-4951-921d-dc49b004b4dc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:38.050149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 348/697] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:38 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:38.051004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886c3c1-202c-437e-ad75-d1b052508dc7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:38.081764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 350/698] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:38.083150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccfe44-a68c-40a9-ad25-8d8fb459eaba None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.087902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccfe44-a68c-40a9-ad25-8d8fb459eaba None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.087902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccfe44-a68c-40a9-ad25-8d8fb459eaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.118559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4fccfe44-a68c-40a9-ad25-8d8fb459eaba None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:38.121065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4fccfe44-a68c-40a9-ad25-8d8fb459eaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:38.179416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:38.201854 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886c3c1-202c-437e-ad75-d1b052508dc7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:38.202813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 351/699] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:38.203992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccfe44-a68c-40a9-ad25-8d8fb459eaba None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:38.241893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6ee6-a9ef-4955-a7bb-9fc919601684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:38.242751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6ee6-a9ef-4955-a7bb-9fc919601684 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.243133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6ee6-a9ef-4955-a7bb-9fc919601684 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.243741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6ee6-a9ef-4955-a7bb-9fc919601684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.319729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:38.352754 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f6ee6-a9ef-4955-a7bb-9fc919601684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:38.353047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccfe44-a68c-40a9-ad25-8d8fb459eaba None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:38.354574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f6ee6-a9ef-4955-a7bb-9fc919601684 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:38.424415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f5467-c90a-458d-9d0e-e77ee11ab717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:38.425216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5467-c90a-458d-9d0e-e77ee11ab717 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.425596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5467-c90a-458d-9d0e-e77ee11ab717 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.425970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5467-c90a-458d-9d0e-e77ee11ab717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.426928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f5467-c90a-458d-9d0e-e77ee11ab717 None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=3384c24dd081414bbf0a2e3d9f71126b) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:38.427203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f5467-c90a-458d-9d0e-e77ee11ab717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:38.429765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f5467-c90a-458d-9d0e-e77ee11ab717 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:38.443066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 352/700] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:38 2022] GET /identity/v3/role_assignments?user.id=3384c24dd081414bbf0a2e3d9f71126b&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:38.444572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7473dbb-e9fe-4a9d-8db9-cbd10d8614fc None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.444942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7473dbb-e9fe-4a9d-8db9-cbd10d8614fc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.445478 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7473dbb-e9fe-4a9d-8db9-cbd10d8614fc None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.453255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 353/701] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:38.454902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630559e1-9f10-4113-a3a9-91ffcea037cb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.455193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630559e1-9f10-4113-a3a9-91ffcea037cb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.455470 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630559e1-9f10-4113-a3a9-91ffcea037cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.484015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:38.509766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 349/702] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 11:30:38.517014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-630559e1-9f10-4113-a3a9-91ffcea037cb None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:38.517014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-630559e1-9f10-4113-a3a9-91ffcea037cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:38.521787 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6ee6-a9ef-4955-a7bb-9fc919601684 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:38.630314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 354/703] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:38.631251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630559e1-9f10-4113-a3a9-91ffcea037cb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:38.660210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6ee6-a9ef-4955-a7bb-9fc919601684 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:38.733340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025cfdc-198f-444a-9621-05756ced45c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:38.734941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025cfdc-198f-444a-9621-05756ced45c6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.734941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025cfdc-198f-444a-9621-05756ced45c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.734941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025cfdc-198f-444a-9621-05756ced45c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.746929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:38.754093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025cfdc-198f-444a-9621-05756ced45c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:38.756169 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025cfdc-198f-444a-9621-05756ced45c6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-a025cfdc-198f-444a-9621-05756ced45c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:38.777864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:38.779133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:38.780192 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:38.780192 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:38.780192 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:38.784513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 350/704] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:38.788304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0ed17-3201-4859-836d-4de26ae42ba6 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.792514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0ed17-3201-4859-836d-4de26ae42ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.792514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0ed17-3201-4859-836d-4de26ae42ba6 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.793873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 351/705] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:38.794726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0ed17-3201-4859-836d-4de26ae42ba6 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:38.807897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630559e1-9f10-4113-a3a9-91ffcea037cb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:38.866911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:38.870359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bfd1a-ef3d-4e28-83c0-f291f6afd9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:38.872249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bfd1a-ef3d-4e28-83c0-f291f6afd9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.872753 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bfd1a-ef3d-4e28-83c0-f291f6afd9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.873033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bfd1a-ef3d-4e28-83c0-f291f6afd9b9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.873680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3bfd1a-ef3d-4e28-83c0-f291f6afd9b9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:38.876621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3bfd1a-ef3d-4e28-83c0-f291f6afd9b9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7e3bfd1a-ef3d-4e28-83c0-f291f6afd9b9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:38.895568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:38.896749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:38.897606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 355/706] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Aug 31 11:30:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:38.898944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341412c-0837-409a-8d9d-b2a4f9407d1d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:38.898944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341412c-0837-409a-8d9d-b2a4f9407d1d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:38.899229 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341412c-0837-409a-8d9d-b2a4f9407d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:38.933729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2341412c-0837-409a-8d9d-b2a4f9407d1d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:38.934877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2341412c-0837-409a-8d9d-b2a4f9407d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:38.967004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0ed17-3201-4859-836d-4de26ae42ba6 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:39.011784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 356/707] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:39.013001 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341412c-0837-409a-8d9d-b2a4f9407d1d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:39.025206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b454ae2-0b92-45bc-9331-c52608db32e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:39.026869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b454ae2-0b92-45bc-9331-c52608db32e7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:39.027321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b454ae2-0b92-45bc-9331-c52608db32e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:39.028431 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b454ae2-0b92-45bc-9331-c52608db32e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:39.029987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b454ae2-0b92-45bc-9331-c52608db32e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:39.030482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b454ae2-0b92-45bc-9331-c52608db32e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:39.032984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b454ae2-0b92-45bc-9331-c52608db32e7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:39.056518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 352/708] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:39.056518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b454ae2-0b92-45bc-9331-c52608db32e7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:39.104062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:39.144628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341412c-0837-409a-8d9d-b2a4f9407d1d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:39.185410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e000f-edf3-4969-adc2-4851be75c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:39.186615 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e000f-edf3-4969-adc2-4851be75c500 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:39.187099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e000f-edf3-4969-adc2-4851be75c500 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:39.187572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e000f-edf3-4969-adc2-4851be75c500 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:39.188427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e000f-edf3-4969-adc2-4851be75c500 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:39.189102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e000f-edf3-4969-adc2-4851be75c500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:39.190661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e000f-edf3-4969-adc2-4851be75c500 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:39.196752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 357/709] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Aug 31 11:30:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:39.198443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e000f-edf3-4969-adc2-4851be75c500 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:39.245966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:39.254925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b454ae2-0b92-45bc-9331-c52608db32e7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:39.311945 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e322e6-6ddc-472b-8ca0-4aa0b76f7cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:39.314339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e322e6-6ddc-472b-8ca0-4aa0b76f7cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:39.314789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e322e6-6ddc-472b-8ca0-4aa0b76f7cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:39.315227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e322e6-6ddc-472b-8ca0-4aa0b76f7cb7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:39.316862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e322e6-6ddc-472b-8ca0-4aa0b76f7cb7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:39.318445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e322e6-6ddc-472b-8ca0-4aa0b76f7cb7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:39.320076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e000f-edf3-4969-adc2-4851be75c500 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-f9e322e6-6ddc-472b-8ca0-4aa0b76f7cb7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:39.334517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:39.335538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:39.336378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 353/710] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:39 2022] GET /identity/v3/services/network => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:39.337214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e322e6-6ddc-472b-8ca0-4aa0b76f7cb7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:39.363409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b316d1-6c2f-4228-8d93-54a24a7aea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:39.364379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b316d1-6c2f-4228-8d93-54a24a7aea6f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:39.364950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b316d1-6c2f-4228-8d93-54a24a7aea6f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:39.365447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b316d1-6c2f-4228-8d93-54a24a7aea6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:39.367826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:39.376072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b316d1-6c2f-4228-8d93-54a24a7aea6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:39.377803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b316d1-6c2f-4228-8d93-54a24a7aea6f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-01b316d1-6c2f-4228-8d93-54a24a7aea6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:39.391044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:39.392126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:39.393167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:39.393167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:39.393167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:39.393167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:39.393167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:39.393167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:39.393167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:39.393167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:39.400009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 358/711] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:39.401155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b316d1-6c2f-4228-8d93-54a24a7aea6f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:39.461909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e322e6-6ddc-472b-8ca0-4aa0b76f7cb7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:39.518298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a06e5-1470-4b6f-9502-c37c4074ff0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:39.518921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a06e5-1470-4b6f-9502-c37c4074ff0b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:39.519475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a06e5-1470-4b6f-9502-c37c4074ff0b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:39.519755 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a06e5-1470-4b6f-9502-c37c4074ff0b None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:39.521665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a06e5-1470-4b6f-9502-c37c4074ff0b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:39.522883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a06e5-1470-4b6f-9502-c37c4074ff0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:39.527793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a06e5-1470-4b6f-9502-c37c4074ff0b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:39.533765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b316d1-6c2f-4228-8d93-54a24a7aea6f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:39.534932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 354/712] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Aug 31 11:30:39 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:39.535759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a06e5-1470-4b6f-9502-c37c4074ff0b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:39.566603 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:39.580855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f077e-8d32-44fe-8fd6-7ac65241553d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:39.581910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f077e-8d32-44fe-8fd6-7ac65241553d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:39.582465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f077e-8d32-44fe-8fd6-7ac65241553d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:39.582945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f077e-8d32-44fe-8fd6-7ac65241553d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:39.596643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f077e-8d32-44fe-8fd6-7ac65241553d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:39.598635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f077e-8d32-44fe-8fd6-7ac65241553d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d42f077e-8d32-44fe-8fd6-7ac65241553d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:39.618102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:39.619222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:39.620282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:39.620282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:39.620282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:39.624029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 359/713] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:39.625341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f077e-8d32-44fe-8fd6-7ac65241553d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:39.662574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a06e5-1470-4b6f-9502-c37c4074ff0b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:39.693210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:39.729677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb33ef-d5c4-4a2c-9b68-a98755c22eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:39.731480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb33ef-d5c4-4a2c-9b68-a98755c22eea None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:39.731480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb33ef-d5c4-4a2c-9b68-a98755c22eea None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:39.731480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb33ef-d5c4-4a2c-9b68-a98755c22eea None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:39.731918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb33ef-d5c4-4a2c-9b68-a98755c22eea None admin] RBAC: Adding query filter params (name=network) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:39.732228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb33ef-d5c4-4a2c-9b68-a98755c22eea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:39.734152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb33ef-d5c4-4a2c-9b68-a98755c22eea None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:39.741798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 355/714] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:30:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:39.742448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb33ef-d5c4-4a2c-9b68-a98755c22eea None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:39.787644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f077e-8d32-44fe-8fd6-7ac65241553d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:39.863261 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8510a-323d-446a-926a-36c74deb62dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:39.863874 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8510a-323d-446a-926a-36c74deb62dd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:39.864150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8510a-323d-446a-926a-36c74deb62dd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:39.864416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8510a-323d-446a-926a-36c74deb62dd None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:39.865239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8510a-323d-446a-926a-36c74deb62dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:39.865541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8510a-323d-446a-926a-36c74deb62dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:39.868643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8510a-323d-446a-926a-36c74deb62dd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:39.901041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 360/715] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:39.901041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8510a-323d-446a-926a-36c74deb62dd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:39.911621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:39.961675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb33ef-d5c4-4a2c-9b68-a98755c22eea None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:40.030654 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e694f4-35ac-4b32-86cb-026a09cfd5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:40.031289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e694f4-35ac-4b32-86cb-026a09cfd5ba None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:40.031517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e694f4-35ac-4b32-86cb-026a09cfd5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:40.031729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e694f4-35ac-4b32-86cb-026a09cfd5ba None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:40.032386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e694f4-35ac-4b32-86cb-026a09cfd5ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:40.033690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e694f4-35ac-4b32-86cb-026a09cfd5ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:40.035065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e694f4-35ac-4b32-86cb-026a09cfd5ba None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:40.068478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 356/716] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:40.069501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e694f4-35ac-4b32-86cb-026a09cfd5ba None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:40.102471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8510a-323d-446a-926a-36c74deb62dd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:40.112444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:40.151885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb28b70-c579-49b3-9916-4e6b93aaae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:40.153028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb28b70-c579-49b3-9916-4e6b93aaae72 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:40.153515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb28b70-c579-49b3-9916-4e6b93aaae72 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:40.154190 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb28b70-c579-49b3-9916-4e6b93aaae72 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:40.155322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb28b70-c579-49b3-9916-4e6b93aaae72 None admin] RBAC: Adding query filter params (type=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:40.155878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb28b70-c579-49b3-9916-4e6b93aaae72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:40.157801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb28b70-c579-49b3-9916-4e6b93aaae72 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:40.166203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 361/717] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:30:39 2022] GET /identity/v3/services?type=network => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:40.167381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb28b70-c579-49b3-9916-4e6b93aaae72 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:40.272896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e694f4-35ac-4b32-86cb-026a09cfd5ba None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:40.378545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb28b70-c579-49b3-9916-4e6b93aaae72 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:40.425695 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7306de5-cff8-407b-adc7-7529dc20ec20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:40.426225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7306de5-cff8-407b-adc7-7529dc20ec20 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:40.426506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7306de5-cff8-407b-adc7-7529dc20ec20 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:40.426774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7306de5-cff8-407b-adc7-7529dc20ec20 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:40.427410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7306de5-cff8-407b-adc7-7529dc20ec20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:40.431232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7306de5-cff8-407b-adc7-7529dc20ec20 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:40.453223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ce180-0020-446f-95c9-370bd48892cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:40.454795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ce180-0020-446f-95c9-370bd48892cb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:40.454795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ce180-0020-446f-95c9-370bd48892cb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:40.454795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ce180-0020-446f-95c9-370bd48892cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:40.455353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ce180-0020-446f-95c9-370bd48892cb None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=f4048316ea784c389eb249b4dd915e0e) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:40.455575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ce180-0020-446f-95c9-370bd48892cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:40.457083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ce180-0020-446f-95c9-370bd48892cb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:40.473000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 357/718] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:40 2022] GET /identity/v3/role_assignments?user.id=f4048316ea784c389eb249b4dd915e0e&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 259 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:40.491136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 362/719] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:30:40 2022] POST /identity/v3/users => generated 254 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:30:40.657736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bd3b1-a8a3-4519-8e82-1feb17e797a3 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:40.658031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bd3b1-a8a3-4519-8e82-1feb17e797a3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:40.658269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bd3b1-a8a3-4519-8e82-1feb17e797a3 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:40.659688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 358/720] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:40.673513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488b36d-2232-49e0-a04e-6d3d06236119 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:40.673815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488b36d-2232-49e0-a04e-6d3d06236119 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:40.677655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488b36d-2232-49e0-a04e-6d3d06236119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:40.714652 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1488b36d-2232-49e0-a04e-6d3d06236119 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:40.716519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1488b36d-2232-49e0-a04e-6d3d06236119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:40.747814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:40.818716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 363/721] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:41.298353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bd3b1-a8a3-4519-8e82-1feb17e797a3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:41.320427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:41.545281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bd3b1-a8a3-4519-8e82-1feb17e797a3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:41.608092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bb54e-0905-46b1-b203-17a51c0ade96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:41.608757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bb54e-0905-46b1-b203-17a51c0ade96 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:41.609037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bb54e-0905-46b1-b203-17a51c0ade96 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:41.609309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bb54e-0905-46b1-b203-17a51c0ade96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:41.630098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bb54e-0905-46b1-b203-17a51c0ade96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-aa4bb54e-0905-46b1-b203-17a51c0ade96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:41.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:41.636927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:41.638365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 359/722] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:41.642661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488b36d-2232-49e0-a04e-6d3d06236119 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:41.774812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488b36d-2232-49e0-a04e-6d3d06236119 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:41.811806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db561179-0d63-4e60-a260-c411263b4e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:41.812413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db561179-0d63-4e60-a260-c411263b4e5d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:41.812721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db561179-0d63-4e60-a260-c411263b4e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:41.812996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db561179-0d63-4e60-a260-c411263b4e5d None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:41.813654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db561179-0d63-4e60-a260-c411263b4e5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:41.813959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db561179-0d63-4e60-a260-c411263b4e5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:41.815320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db561179-0d63-4e60-a260-c411263b4e5d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:41.825094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 364/723] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:41.830181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bb54e-0905-46b1-b203-17a51c0ade96 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:41.957897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:41.977801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bb54e-0905-46b1-b203-17a51c0ade96 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:42.024233 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71356fd-61ee-47ca-8db0-19b0ac600613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:42.024890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71356fd-61ee-47ca-8db0-19b0ac600613 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:42.025170 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71356fd-61ee-47ca-8db0-19b0ac600613 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:42.025976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71356fd-61ee-47ca-8db0-19b0ac600613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:42.043640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71356fd-61ee-47ca-8db0-19b0ac600613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:42.045157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71356fd-61ee-47ca-8db0-19b0ac600613 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c71356fd-61ee-47ca-8db0-19b0ac600613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:42.062443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:42.063496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:42.064378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:42.064378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:42.064378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:42.068817 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 360/724] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:42.069667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db561179-0d63-4e60-a260-c411263b4e5d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:42.102315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:42.206868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6badee-c500-45b6-9a4e-0952dbee571e None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:42.207212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6badee-c500-45b6-9a4e-0952dbee571e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:42.207428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6badee-c500-45b6-9a4e-0952dbee571e None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:42.209885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 361/725] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:42.223231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f293d7c-a8f4-4ca8-aaf8-d6739dcf2d08 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:42.223883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f293d7c-a8f4-4ca8-aaf8-d6739dcf2d08 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:42.224727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f293d7c-a8f4-4ca8-aaf8-d6739dcf2d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:42.256872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7f293d7c-a8f4-4ca8-aaf8-d6739dcf2d08 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:42.262199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7f293d7c-a8f4-4ca8-aaf8-d6739dcf2d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:42.268116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db561179-0d63-4e60-a260-c411263b4e5d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:42.339071 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca979d8-18ce-47ac-a70d-6bccb1e36ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:42.340060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca979d8-18ce-47ac-a70d-6bccb1e36ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:42.340607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca979d8-18ce-47ac-a70d-6bccb1e36ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:42.341128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca979d8-18ce-47ac-a70d-6bccb1e36ec2 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:42.342016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca979d8-18ce-47ac-a70d-6bccb1e36ec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:42.342517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca979d8-18ce-47ac-a70d-6bccb1e36ec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:42.344070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca979d8-18ce-47ac-a70d-6bccb1e36ec2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:42.362366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 365/726] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:42.368256 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca979d8-18ce-47ac-a70d-6bccb1e36ec2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:42.414175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 362/727] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:42.468391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:42.496088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca979d8-18ce-47ac-a70d-6bccb1e36ec2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:42.532310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692da2c2-e5a9-450c-81f3-86f7d6508baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:42.532660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692da2c2-e5a9-450c-81f3-86f7d6508baa None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:42.533880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692da2c2-e5a9-450c-81f3-86f7d6508baa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:42.533880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692da2c2-e5a9-450c-81f3-86f7d6508baa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:42.551746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692da2c2-e5a9-450c-81f3-86f7d6508baa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:42.553218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692da2c2-e5a9-450c-81f3-86f7d6508baa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-692da2c2-e5a9-450c-81f3-86f7d6508baa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:42.578197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:42.579170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:42.580102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:42.580102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:42.580102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:42.580102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:42.580102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:42.580102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:42.580102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:42.580102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:42.580102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:42.580102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:42.580102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 366/728] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:42.585580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f293d7c-a8f4-4ca8-aaf8-d6739dcf2d08 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:42.713531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:42.756953 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ea53c-b52d-466c-a494-3bd0dcf05021 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:42.757453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ea53c-b52d-466c-a494-3bd0dcf05021 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:42.757947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ea53c-b52d-466c-a494-3bd0dcf05021 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:42.761707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 367/729] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:42.766337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be6a4-623c-4c7e-ae99-8dee4193fc98 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:42.766852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be6a4-623c-4c7e-ae99-8dee4193fc98 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:42.767368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be6a4-623c-4c7e-ae99-8dee4193fc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:42.799364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6e0be6a4-623c-4c7e-ae99-8dee4193fc98 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:42.801824 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6e0be6a4-623c-4c7e-ae99-8dee4193fc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:42.815912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f293d7c-a8f4-4ca8-aaf8-d6739dcf2d08 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:42.925396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 368/730] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:42.926979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be6a4-623c-4c7e-ae99-8dee4193fc98 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:42.945268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c2f03-35f4-4191-a800-35bcddaaedcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:42.946255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2f03-35f4-4191-a800-35bcddaaedcb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:42.946755 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2f03-35f4-4191-a800-35bcddaaedcb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:42.947461 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2f03-35f4-4191-a800-35bcddaaedcb None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:42.948414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c2f03-35f4-4191-a800-35bcddaaedcb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:42.948947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c2f03-35f4-4191-a800-35bcddaaedcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:42.956579 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c2f03-35f4-4191-a800-35bcddaaedcb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:43.013040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 363/731] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Aug 31 11:30:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:43.018106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c2f03-35f4-4191-a800-35bcddaaedcb None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:43.099182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:43.138666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be6a4-623c-4c7e-ae99-8dee4193fc98 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:43.184775 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8e0a0-1268-4252-9c28-49fcfa75f12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:43.185395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8e0a0-1268-4252-9c28-49fcfa75f12f None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:43.185673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8e0a0-1268-4252-9c28-49fcfa75f12f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:43.185939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8e0a0-1268-4252-9c28-49fcfa75f12f None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:43.186595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8e0a0-1268-4252-9c28-49fcfa75f12f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:43.187956 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8e0a0-1268-4252-9c28-49fcfa75f12f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:43.192625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c2f03-35f4-4191-a800-35bcddaaedcb None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:43.206326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:43.214227 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 369/732] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:30:42 2022] POST /identity/v3/services => generated 251 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:30:43.216069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba70b2-0a8d-4801-8cdf-c97da91ec6db None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:43.216360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba70b2-0a8d-4801-8cdf-c97da91ec6db None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:43.216774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba70b2-0a8d-4801-8cdf-c97da91ec6db None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:43.218355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 370/733] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:43.226180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194a818-d4c2-42f4-beed-3e6fd5b367e0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:43.226490 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194a818-d4c2-42f4-beed-3e6fd5b367e0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:43.226717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194a818-d4c2-42f4-beed-3e6fd5b367e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:43.248073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2949bf-82da-4ae1-88b4-9ad8978a45b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:43.248948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2949bf-82da-4ae1-88b4-9ad8978a45b0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:43.249235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2949bf-82da-4ae1-88b4-9ad8978a45b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:43.249508 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2949bf-82da-4ae1-88b4-9ad8978a45b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:43.274250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2194a818-d4c2-42f4-beed-3e6fd5b367e0 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:43.277005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2194a818-d4c2-42f4-beed-3e6fd5b367e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:43.338352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2949bf-82da-4ae1-88b4-9ad8978a45b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:43.339963 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2949bf-82da-4ae1-88b4-9ad8978a45b0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:43.363733 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 371/734] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:43 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 11:30:43.365670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194a818-d4c2-42f4-beed-3e6fd5b367e0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:43.436959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:43.465564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 364/735] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:43 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 11:30:43.470631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2949bf-82da-4ae1-88b4-9ad8978a45b0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:43.481280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:43.491140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194a818-d4c2-42f4-beed-3e6fd5b367e0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:43.540184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ab44b-0c74-4633-b62b-e28dcdef2180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:43.541986 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ab44b-0c74-4633-b62b-e28dcdef2180 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:43.542873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ab44b-0c74-4633-b62b-e28dcdef2180 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:43.543310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ab44b-0c74-4633-b62b-e28dcdef2180 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:43.544391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ab44b-0c74-4633-b62b-e28dcdef2180 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:43.546769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ab44b-0c74-4633-b62b-e28dcdef2180 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:43.574342 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 372/736] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:30:43 2022] POST /identity/v3/services => generated 240 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:30:43.598877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2949bf-82da-4ae1-88b4-9ad8978a45b0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:43.617738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d17d0-e234-4c7b-abab-e6ea5d6d46b0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:43.618089 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d17d0-e234-4c7b-abab-e6ea5d6d46b0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:43.618359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d17d0-e234-4c7b-abab-e6ea5d6d46b0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:43.621085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 373/737] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:30:43.627750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a62e7-4d75-430c-91fb-24d6bce2bb9c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:43.628073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a62e7-4d75-430c-91fb-24d6bce2bb9c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:43.628350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a62e7-4d75-430c-91fb-24d6bce2bb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:43.660672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1f9a62e7-4d75-430c-91fb-24d6bce2bb9c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:43.662022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1f9a62e7-4d75-430c-91fb-24d6bce2bb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:43.673330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cb901-2a60-4b80-ab6c-5754e9fb19ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:43.676804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cb901-2a60-4b80-ab6c-5754e9fb19ac None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:43.677139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cb901-2a60-4b80-ab6c-5754e9fb19ac None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:43.677564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cb901-2a60-4b80-ab6c-5754e9fb19ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:43.691751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8cb901-2a60-4b80-ab6c-5754e9fb19ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:43.693647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8cb901-2a60-4b80-ab6c-5754e9fb19ac None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-fb8cb901-2a60-4b80-ab6c-5754e9fb19ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:43.710201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:43.711427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:43.712555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:43.712555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:43.712555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:43.714923 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 365/738] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:43.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cb901-2a60-4b80-ab6c-5754e9fb19ac None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:43.751545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 374/739] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:30:43.756545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a62e7-4d75-430c-91fb-24d6bce2bb9c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:43.784935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:43.813155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:43.848637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cb901-2a60-4b80-ab6c-5754e9fb19ac None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:43.880958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a62e7-4d75-430c-91fb-24d6bce2bb9c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:43.892795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57890f0f-6136-48b4-8048-0e03221cc6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:43.894040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57890f0f-6136-48b4-8048-0e03221cc6bf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:43.894548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57890f0f-6136-48b4-8048-0e03221cc6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:43.895033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57890f0f-6136-48b4-8048-0e03221cc6bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:43.918808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57890f0f-6136-48b4-8048-0e03221cc6bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-57890f0f-6136-48b4-8048-0e03221cc6bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:43.925774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:43.926877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:43.927866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:43.932102 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 366/740] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:43.940699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57890f0f-6136-48b4-8048-0e03221cc6bf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:43.953392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa6d1b-72ff-45de-8903-1298b7b0997a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:43.954661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6d1b-72ff-45de-8903-1298b7b0997a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:43.956548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6d1b-72ff-45de-8903-1298b7b0997a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:43.956970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6d1b-72ff-45de-8903-1298b7b0997a None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:43.957780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa6d1b-72ff-45de-8903-1298b7b0997a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:43.959012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa6d1b-72ff-45de-8903-1298b7b0997a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:43.961251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa6d1b-72ff-45de-8903-1298b7b0997a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:43.977879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 375/741] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:43.981055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa6d1b-72ff-45de-8903-1298b7b0997a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:44.121187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57890f0f-6136-48b4-8048-0e03221cc6bf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:44.132250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa6d1b-72ff-45de-8903-1298b7b0997a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:44.137686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:44.143411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:44.159719 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fc938-ff02-456e-bb93-ae8cf8bad54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:44.160587 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fc938-ff02-456e-bb93-ae8cf8bad54e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:44.161046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fc938-ff02-456e-bb93-ae8cf8bad54e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:44.161468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fc938-ff02-456e-bb93-ae8cf8bad54e None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:44.162283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fc938-ff02-456e-bb93-ae8cf8bad54e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:44.162735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fc938-ff02-456e-bb93-ae8cf8bad54e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:44.164250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fc938-ff02-456e-bb93-ae8cf8bad54e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:44.171794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a028f9a-fcd3-45d4-ac55-ff4a474ffd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:44.172482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028f9a-fcd3-45d4-ac55-ff4a474ffd94 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:44.172785 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028f9a-fcd3-45d4-ac55-ff4a474ffd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:44.173059 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028f9a-fcd3-45d4-ac55-ff4a474ffd94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:44.174310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 367/742] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:44.179585 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fc938-ff02-456e-bb93-ae8cf8bad54e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:44.180358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a028f9a-fcd3-45d4-ac55-ff4a474ffd94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:44.182459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a028f9a-fcd3-45d4-ac55-ff4a474ffd94 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7a028f9a-fcd3-45d4-ac55-ff4a474ffd94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:44.194663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:44.195772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:44.196786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:44.196786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:44.196786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:44.196786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:44.196786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:44.196786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:44.196786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:44.196786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:44.196786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 376/743] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:44.197871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a028f9a-fcd3-45d4-ac55-ff4a474ffd94 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:44.295876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fc938-ff02-456e-bb93-ae8cf8bad54e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:44.309348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a028f9a-fcd3-45d4-ac55-ff4a474ffd94 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:44.331742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8b783-e6b5-4ce1-ae8a-9960350c5415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:44.332375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8b783-e6b5-4ce1-ae8a-9960350c5415 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:44.332662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8b783-e6b5-4ce1-ae8a-9960350c5415 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:44.332944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8b783-e6b5-4ce1-ae8a-9960350c5415 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:44.343870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8b783-e6b5-4ce1-ae8a-9960350c5415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-93e8b783-e6b5-4ce1-ae8a-9960350c5415 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:44.349444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:44.350366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:44.351229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:44.355577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 368/744] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:44.356420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8b783-e6b5-4ce1-ae8a-9960350c5415 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:44.359563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21e0f6-760e-425c-9ea5-be9b41094435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:44.360381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21e0f6-760e-425c-9ea5-be9b41094435 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:44.360695 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21e0f6-760e-425c-9ea5-be9b41094435 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:44.360971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21e0f6-760e-425c-9ea5-be9b41094435 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:44.376307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21e0f6-760e-425c-9ea5-be9b41094435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:44.377887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21e0f6-760e-425c-9ea5-be9b41094435 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:44.395349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9b21e0f6-760e-425c-9ea5-be9b41094435 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:44.396729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:44.397702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:44.398608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:44.398608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:44.398608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:44.401851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 377/745] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:44.406028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21e0f6-760e-425c-9ea5-be9b41094435 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:44.431356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:44.493571 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8b783-e6b5-4ce1-ae8a-9960350c5415 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:44.523279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21e0f6-760e-425c-9ea5-be9b41094435 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:44.539453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768cdf5-79ef-4c70-8a8a-1e76c48fbbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:44.540475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768cdf5-79ef-4c70-8a8a-1e76c48fbbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:44.541688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768cdf5-79ef-4c70-8a8a-1e76c48fbbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:44.542173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768cdf5-79ef-4c70-8a8a-1e76c48fbbb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:44.543091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768cdf5-79ef-4c70-8a8a-1e76c48fbbb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:44.543621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768cdf5-79ef-4c70-8a8a-1e76c48fbbb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:44.545371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768cdf5-79ef-4c70-8a8a-1e76c48fbbb6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:44.560339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 369/746] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:44.561695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768cdf5-79ef-4c70-8a8a-1e76c48fbbb6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:44.682605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:44.691060 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768cdf5-79ef-4c70-8a8a-1e76c48fbbb6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:44.691693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e382-c8a3-4366-9035-f3c57566da03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:44.692375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e382-c8a3-4366-9035-f3c57566da03 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:44.692791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e382-c8a3-4366-9035-f3c57566da03 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:44.693129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e382-c8a3-4366-9035-f3c57566da03 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:44.693920 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514e382-c8a3-4366-9035-f3c57566da03 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:44.694233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514e382-c8a3-4366-9035-f3c57566da03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:44.695732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514e382-c8a3-4366-9035-f3c57566da03 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:44.709424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 378/747] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:44.710620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e382-c8a3-4366-9035-f3c57566da03 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:44.739037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c1b75-bec0-40ea-b54a-4253804643e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:44.740207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c1b75-bec0-40ea-b54a-4253804643e2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:44.741131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c1b75-bec0-40ea-b54a-4253804643e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:44.741726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c1b75-bec0-40ea-b54a-4253804643e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:44.742653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c1b75-bec0-40ea-b54a-4253804643e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:44.743430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c1b75-bec0-40ea-b54a-4253804643e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:44.745857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c1b75-bec0-40ea-b54a-4253804643e2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:44.765666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 370/748] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:44.765666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c1b75-bec0-40ea-b54a-4253804643e2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:44.817758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:44.837935 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e382-c8a3-4366-9035-f3c57566da03 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:44.872608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43966c45-d842-4128-b699-92a431a722b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:44.873328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966c45-d842-4128-b699-92a431a722b2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:44.873532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966c45-d842-4128-b699-92a431a722b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:44.873768 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966c45-d842-4128-b699-92a431a722b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:44.874517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43966c45-d842-4128-b699-92a431a722b2 None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=3384c24dd081414bbf0a2e3d9f71126b) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:44.874826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43966c45-d842-4128-b699-92a431a722b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:44.876321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43966c45-d842-4128-b699-92a431a722b2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:44.882845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c1b75-bec0-40ea-b54a-4253804643e2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:44.883047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 379/749] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:44 2022] GET /identity/v3/role_assignments?user.id=3384c24dd081414bbf0a2e3d9f71126b&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:44.883961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43966c45-d842-4128-b699-92a431a722b2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:44.955705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2ddd4-2a7b-471d-9ad2-ecb5c851bdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:44.956863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2ddd4-2a7b-471d-9ad2-ecb5c851bdb0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:44.957355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2ddd4-2a7b-471d-9ad2-ecb5c851bdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:44.957854 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2ddd4-2a7b-471d-9ad2-ecb5c851bdb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:44.978619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2ddd4-2a7b-471d-9ad2-ecb5c851bdb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:44.981420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2ddd4-2a7b-471d-9ad2-ecb5c851bdb0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:44.996751 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43966c45-d842-4128-b699-92a431a722b2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-34d2ddd4-2a7b-471d-9ad2-ecb5c851bdb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:45.024321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:45.025953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:45.026888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:45.026888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:45.026888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:45.026888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 371/750] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:45.032781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2ddd4-2a7b-471d-9ad2-ecb5c851bdb0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:45.047930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:45.086236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d165439-9dad-4e01-a6a5-4fc0ee4a839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:45.087293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d165439-9dad-4e01-a6a5-4fc0ee4a839d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.088250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d165439-9dad-4e01-a6a5-4fc0ee4a839d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.089841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d165439-9dad-4e01-a6a5-4fc0ee4a839d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.108526 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d165439-9dad-4e01-a6a5-4fc0ee4a839d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:45.112228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d165439-9dad-4e01-a6a5-4fc0ee4a839d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6d165439-9dad-4e01-a6a5-4fc0ee4a839d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:45.136389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:45.137550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:45.138483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:45.138483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:45.138483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:45.138483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:45.138483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:45.138483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:45.138483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:45.138483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:45.138483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:45.138483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:45.141657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 380/751] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Aug 31 11:30:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:45.148246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d165439-9dad-4e01-a6a5-4fc0ee4a839d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:45.164151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:45.247201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2ddd4-2a7b-471d-9ad2-ecb5c851bdb0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:45.290272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fad9c-e6a3-496d-8a08-eff137f4c488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:45.296036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fad9c-e6a3-496d-8a08-eff137f4c488 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.296584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fad9c-e6a3-496d-8a08-eff137f4c488 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.297102 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fad9c-e6a3-496d-8a08-eff137f4c488 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.297959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fad9c-e6a3-496d-8a08-eff137f4c488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:45.298488 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fad9c-e6a3-496d-8a08-eff137f4c488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:45.300068 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fad9c-e6a3-496d-8a08-eff137f4c488 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:45.326479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 372/752] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:45.333377 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fad9c-e6a3-496d-8a08-eff137f4c488 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:45.355670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d165439-9dad-4e01-a6a5-4fc0ee4a839d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:45.426160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fd566-8d37-4a4f-b7e2-6ebd3ad17614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:45.426784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fd566-8d37-4a4f-b7e2-6ebd3ad17614 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.427048 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fd566-8d37-4a4f-b7e2-6ebd3ad17614 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.427320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fd566-8d37-4a4f-b7e2-6ebd3ad17614 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.428053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fd566-8d37-4a4f-b7e2-6ebd3ad17614 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:45.428354 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fd566-8d37-4a4f-b7e2-6ebd3ad17614 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:45.429803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fd566-8d37-4a4f-b7e2-6ebd3ad17614 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:45.437139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:45.466624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 381/753] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Aug 31 11:30:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:45.471964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fd566-8d37-4a4f-b7e2-6ebd3ad17614 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:45.476122 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fad9c-e6a3-496d-8a08-eff137f4c488 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:45.520733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a0a76-d3f0-47bf-9e74-2cbfc536eabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:45.522135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a0a76-d3f0-47bf-9e74-2cbfc536eabc None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.522750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a0a76-d3f0-47bf-9e74-2cbfc536eabc None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.523877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a0a76-d3f0-47bf-9e74-2cbfc536eabc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.534137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a0a76-d3f0-47bf-9e74-2cbfc536eabc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:45.535943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a0a76-d3f0-47bf-9e74-2cbfc536eabc None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d24a0a76-d3f0-47bf-9e74-2cbfc536eabc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:45.551115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:45.552139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:45.553451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:45.553451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:45.553451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:45.553451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:45.553451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:45.553451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:45.553451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:45.553451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:45.553451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:30:45.553451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:45.555954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 373/754] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Aug 31 11:30:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:45.558201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afa0f3-1304-4613-9ea7-dc91eb7ef9ae None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.558703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afa0f3-1304-4613-9ea7-dc91eb7ef9ae None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.559187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afa0f3-1304-4613-9ea7-dc91eb7ef9ae None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.561013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 374/755] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:45.561976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afa0f3-1304-4613-9ea7-dc91eb7ef9ae None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:45.569537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:45.608902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fd566-8d37-4a4f-b7e2-6ebd3ad17614 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:45.645096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ee5b7-98a6-43da-b026-900e47fc6971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:45.645096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee5b7-98a6-43da-b026-900e47fc6971 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.645384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee5b7-98a6-43da-b026-900e47fc6971 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.645635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee5b7-98a6-43da-b026-900e47fc6971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.656307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ee5b7-98a6-43da-b026-900e47fc6971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:45.657860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ee5b7-98a6-43da-b026-900e47fc6971 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ea1ee5b7-98a6-43da-b026-900e47fc6971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:45.672913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:45.673926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:45.674886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:45.674886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:45.674886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:45.674886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 382/756] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:45.678347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb70ee-75c9-4291-af4e-0901648a0cb5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.678347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb70ee-75c9-4291-af4e-0901648a0cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.678347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb70ee-75c9-4291-af4e-0901648a0cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.693307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afa0f3-1304-4613-9ea7-dc91eb7ef9ae None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:45.700618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5fcb70ee-75c9-4291-af4e-0901648a0cb5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:45.702396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5fcb70ee-75c9-4291-af4e-0901648a0cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:45.731608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d94ed9-0167-47de-beef-947b7039b0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:45.732578 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d94ed9-0167-47de-beef-947b7039b0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.733006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d94ed9-0167-47de-beef-947b7039b0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.733509 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d94ed9-0167-47de-beef-947b7039b0a2 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.734514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d94ed9-0167-47de-beef-947b7039b0a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:45.734965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d94ed9-0167-47de-beef-947b7039b0a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:45.736676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d94ed9-0167-47de-beef-947b7039b0a2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:45.760879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:45.761473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 375/757] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Aug 31 11:30:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:45.762337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d94ed9-0167-47de-beef-947b7039b0a2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:45.786594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 383/758] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:30:45.787836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb70ee-75c9-4291-af4e-0901648a0cb5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:45.834760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:45.874490 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d94ed9-0167-47de-beef-947b7039b0a2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:45.890680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb70ee-75c9-4291-af4e-0901648a0cb5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:45.911803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed695d-49f2-4b46-b0ea-3f3089281dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:45.913574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed695d-49f2-4b46-b0ea-3f3089281dec None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.913864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed695d-49f2-4b46-b0ea-3f3089281dec None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.914140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed695d-49f2-4b46-b0ea-3f3089281dec None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.914788 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed695d-49f2-4b46-b0ea-3f3089281dec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:45.915092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed695d-49f2-4b46-b0ea-3f3089281dec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:45.916595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed695d-49f2-4b46-b0ea-3f3089281dec None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:45.930154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa71fc-d8b1-47c8-a7a1-0adcf331a6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:45.930433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 376/759] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:45.931306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa71fc-d8b1-47c8-a7a1-0adcf331a6cf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.932271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6457f69-f81f-4a1a-aa16-c7c3c34cb7d8 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.932579 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6457f69-f81f-4a1a-aa16-c7c3c34cb7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.932859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6457f69-f81f-4a1a-aa16-c7c3c34cb7d8 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.933205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa71fc-d8b1-47c8-a7a1-0adcf331a6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.933713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa71fc-d8b1-47c8-a7a1-0adcf331a6cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.934325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 377/760] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:45.940975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a82b0-3136-4468-9d7d-fbc0ca0ed39d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:45.941676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a82b0-3136-4468-9d7d-fbc0ca0ed39d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:45.942106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a82b0-3136-4468-9d7d-fbc0ca0ed39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:45.962500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c38a82b0-3136-4468-9d7d-fbc0ca0ed39d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:45.964169 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c38a82b0-3136-4468-9d7d-fbc0ca0ed39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:46.011534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa71fc-d8b1-47c8-a7a1-0adcf331a6cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:46.013024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa71fc-d8b1-47c8-a7a1-0adcf331a6cf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:46.014419 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:46.048720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 378/761] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:30:46.048720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a82b0-3136-4468-9d7d-fbc0ca0ed39d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:46.112744 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:46.163897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a82b0-3136-4468-9d7d-fbc0ca0ed39d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:46.179378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 384/762] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:30:46.180429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa71fc-d8b1-47c8-a7a1-0adcf331a6cf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:46.205192 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07c26a-0941-42ea-a394-d8e82acee6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:46.205963 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07c26a-0941-42ea-a394-d8e82acee6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:46.206339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07c26a-0941-42ea-a394-d8e82acee6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:46.206708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07c26a-0941-42ea-a394-d8e82acee6c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:46.213826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07c26a-0941-42ea-a394-d8e82acee6c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:46.215460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07c26a-0941-42ea-a394-d8e82acee6c8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-ce07c26a-0941-42ea-a394-d8e82acee6c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:46.226716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:46.227830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:46.229723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:46.229723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:46.229723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:46.229723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:46.229723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:46.229723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:46.229723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:46.229723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:46.236129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 379/763] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:46.236129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07c26a-0941-42ea-a394-d8e82acee6c8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:46.270557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:46.320147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa71fc-d8b1-47c8-a7a1-0adcf331a6cf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:46.351684 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07c26a-0941-42ea-a394-d8e82acee6c8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:46.355817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48126-f5dc-4e4c-9997-90461cec8f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:46.356546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48126-f5dc-4e4c-9997-90461cec8f2b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:46.356896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48126-f5dc-4e4c-9997-90461cec8f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:46.357237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48126-f5dc-4e4c-9997-90461cec8f2b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:46.357900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b48126-f5dc-4e4c-9997-90461cec8f2b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:46.359314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b48126-f5dc-4e4c-9997-90461cec8f2b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-70b48126-f5dc-4e4c-9997-90461cec8f2b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:46.368179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:46.369425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 11:30:46.370259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:46.372629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 385/764] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Aug 31 11:30:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:46.373738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48126-f5dc-4e4c-9997-90461cec8f2b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:46.386577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:46.388433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1eb613-9363-4c15-9aaf-f421ec69ce23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:46.389099 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eb613-9363-4c15-9aaf-f421ec69ce23 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:46.389385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eb613-9363-4c15-9aaf-f421ec69ce23 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:46.389666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eb613-9363-4c15-9aaf-f421ec69ce23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:46.401175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1eb613-9363-4c15-9aaf-f421ec69ce23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:46.402815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1eb613-9363-4c15-9aaf-f421ec69ce23 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-7a1eb613-9363-4c15-9aaf-f421ec69ce23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:46.418497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:46.419436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:46.420349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:46.420349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:46.420349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:46.420349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 380/765] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:46.420349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1eb613-9363-4c15-9aaf-f421ec69ce23 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:46.489497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48126-f5dc-4e4c-9997-90461cec8f2b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:46.520285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:46.531051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d00d32-3265-4c3d-98fe-a0da75120108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:46.531659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d00d32-3265-4c3d-98fe-a0da75120108 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:46.531940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d00d32-3265-4c3d-98fe-a0da75120108 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:46.532245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d00d32-3265-4c3d-98fe-a0da75120108 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:46.532945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d00d32-3265-4c3d-98fe-a0da75120108 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:46.533365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d00d32-3265-4c3d-98fe-a0da75120108 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:46.534851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1eb613-9363-4c15-9aaf-f421ec69ce23 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:46.535318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d00d32-3265-4c3d-98fe-a0da75120108 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:46.547127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 386/766] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:46.547955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d00d32-3265-4c3d-98fe-a0da75120108 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:46.578597 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11b22a-e9d7-4959-af9a-22e5a2ce0a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:46.579147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11b22a-e9d7-4959-af9a-22e5a2ce0a29 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:46.579421 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11b22a-e9d7-4959-af9a-22e5a2ce0a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:46.579688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11b22a-e9d7-4959-af9a-22e5a2ce0a29 None admin] PATH_INFO: `/v3/services/network` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:46.580278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11b22a-e9d7-4959-af9a-22e5a2ce0a29 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:46.581670 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11b22a-e9d7-4959-af9a-22e5a2ce0a29 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-aa11b22a-e9d7-4959-af9a-22e5a2ce0a29 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:46.592458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:46.593540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:46.597227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 381/767] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:46 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:46.598035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11b22a-e9d7-4959-af9a-22e5a2ce0a29 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:46.623000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:46.656604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d00d32-3265-4c3d-98fe-a0da75120108 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:46.700439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba187e-ea99-43ff-a268-c72c2c068911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:46.701068 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba187e-ea99-43ff-a268-c72c2c068911 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:46.701349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba187e-ea99-43ff-a268-c72c2c068911 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:46.701623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba187e-ea99-43ff-a268-c72c2c068911 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:46.702248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba187e-ea99-43ff-a268-c72c2c068911 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:46.702560 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba187e-ea99-43ff-a268-c72c2c068911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:46.703910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba187e-ea99-43ff-a268-c72c2c068911 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:46.708463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 387/768] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Aug 31 11:30:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:46.709087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba187e-ea99-43ff-a268-c72c2c068911 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:46.713376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11b22a-e9d7-4959-af9a-22e5a2ce0a29 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:46.751156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8224281-34ff-4cf3-8418-7c34e6e7b49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:46.751788 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8224281-34ff-4cf3-8418-7c34e6e7b49c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:46.752136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8224281-34ff-4cf3-8418-7c34e6e7b49c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:46.752471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8224281-34ff-4cf3-8418-7c34e6e7b49c None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:46.753501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8224281-34ff-4cf3-8418-7c34e6e7b49c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:46.753501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8224281-34ff-4cf3-8418-7c34e6e7b49c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:46.755037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8224281-34ff-4cf3-8418-7c34e6e7b49c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:46.772585 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 382/769] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:46.772585 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8224281-34ff-4cf3-8418-7c34e6e7b49c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:46.792141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:46.822313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba187e-ea99-43ff-a268-c72c2c068911 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:46.857874 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98447c-558e-42c7-89f0-aac4d1503909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:46.858452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98447c-558e-42c7-89f0-aac4d1503909 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:46.858735 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98447c-558e-42c7-89f0-aac4d1503909 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:46.859011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98447c-558e-42c7-89f0-aac4d1503909 None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/f4048316ea784c389eb249b4dd915e0e/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:46.860966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98447c-558e-42c7-89f0-aac4d1503909 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=f4048316ea784c389eb249b4dd915e0e, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:46.862382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98447c-558e-42c7-89f0-aac4d1503909 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:46.864697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:46.876582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 388/770] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:30:46 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/f4048316ea784c389eb249b4dd915e0e/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:30:46.877616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98447c-558e-42c7-89f0-aac4d1503909 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:46.901707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8224281-34ff-4cf3-8418-7c34e6e7b49c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:46.948600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb9832-91ea-4e04-aa59-5354bfa77608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:46.949709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb9832-91ea-4e04-aa59-5354bfa77608 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:46.950142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb9832-91ea-4e04-aa59-5354bfa77608 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:46.950610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb9832-91ea-4e04-aa59-5354bfa77608 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:46.952377 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb9832-91ea-4e04-aa59-5354bfa77608 None admin] RBAC: Adding query filter params (name=network) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:46.952987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb9832-91ea-4e04-aa59-5354bfa77608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:46.955394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb9832-91ea-4e04-aa59-5354bfa77608 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:46.967497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 383/771] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:30:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:46.968700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb9832-91ea-4e04-aa59-5354bfa77608 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:47.031045 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98447c-558e-42c7-89f0-aac4d1503909 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:47.063265 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:47.071131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d953d6-1e85-474b-a411-be5805c35c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:47.071742 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d953d6-1e85-474b-a411-be5805c35c62 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.072573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d953d6-1e85-474b-a411-be5805c35c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.072860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d953d6-1e85-474b-a411-be5805c35c62 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:47.073495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d953d6-1e85-474b-a411-be5805c35c62 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:47.073906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d953d6-1e85-474b-a411-be5805c35c62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:47.075156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d953d6-1e85-474b-a411-be5805c35c62 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:47.081196 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 389/772] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Aug 31 11:30:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:47.082265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d953d6-1e85-474b-a411-be5805c35c62 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:47.098612 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb9832-91ea-4e04-aa59-5354bfa77608 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:47.137607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6bd40-6975-4352-bc3f-6bc3a464cec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:47.138171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6bd40-6975-4352-bc3f-6bc3a464cec0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.138455 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6bd40-6975-4352-bc3f-6bc3a464cec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.138732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6bd40-6975-4352-bc3f-6bc3a464cec0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:47.145715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6bd40-6975-4352-bc3f-6bc3a464cec0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:47.147178 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6bd40-6975-4352-bc3f-6bc3a464cec0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:47.148508 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-35e6bd40-6975-4352-bc3f-6bc3a464cec0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:47.163207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:47.164161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:47.166876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:47.166876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:47.166876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:47.166876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:47.166876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:47.166876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:47.166876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:47.166876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:47.166876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 384/773] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:47.166876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38c611-691a-4185-b558-7f514bc01990 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.166876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38c611-691a-4185-b558-7f514bc01990 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.167494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38c611-691a-4185-b558-7f514bc01990 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:47.168406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 385/774] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:47.168989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38c611-691a-4185-b558-7f514bc01990 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:47.203900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d953d6-1e85-474b-a411-be5805c35c62 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:47.243049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8f1a6-ce83-4786-94fd-db8d8184129b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:47.243654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8f1a6-ce83-4786-94fd-db8d8184129b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.244647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8f1a6-ce83-4786-94fd-db8d8184129b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.245146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8f1a6-ce83-4786-94fd-db8d8184129b None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:47.245450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8f1a6-ce83-4786-94fd-db8d8184129b None admin] RBAC: Adding query filter params (type=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:47.245754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8f1a6-ce83-4786-94fd-db8d8184129b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:47.247150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8f1a6-ce83-4786-94fd-db8d8184129b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:47.256397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 390/775] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:30:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:47.256397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8f1a6-ce83-4786-94fd-db8d8184129b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:47.286577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38c611-691a-4185-b558-7f514bc01990 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:47.324968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423b28c-e189-41f7-95d4-f860a95946ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:47.325580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423b28c-e189-41f7-95d4-f860a95946ab None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.325866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423b28c-e189-41f7-95d4-f860a95946ab None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.326345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423b28c-e189-41f7-95d4-f860a95946ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:47.326847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423b28c-e189-41f7-95d4-f860a95946ab None admin] RBAC: Adding query filter params (interface=public, service_id=d3e89a761ee04ba4a884258a0d3a7fe3, region_id=RegionOne) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:47.327191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423b28c-e189-41f7-95d4-f860a95946ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:47.328541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423b28c-e189-41f7-95d4-f860a95946ab None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:47.333816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:47.342540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 386/776] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Aug 31 11:30:47 2022] GET /identity/v3/endpoints?service_id=d3e89a761ee04ba4a884258a0d3a7fe3&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:47.343627 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820a2ab-e260-4d45-a9ac-af4958cd6428 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.343915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820a2ab-e260-4d45-a9ac-af4958cd6428 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.344193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820a2ab-e260-4d45-a9ac-af4958cd6428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:47.364393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5820a2ab-e260-4d45-a9ac-af4958cd6428 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:47.365639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5820a2ab-e260-4d45-a9ac-af4958cd6428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:47.371647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8f1a6-ce83-4786-94fd-db8d8184129b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:47.395858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:47.410709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904018f1-b4c5-43a7-bd36-57cdad1154d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:47.411587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904018f1-b4c5-43a7-bd36-57cdad1154d0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.411914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904018f1-b4c5-43a7-bd36-57cdad1154d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.412195 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904018f1-b4c5-43a7-bd36-57cdad1154d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:47.412960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904018f1-b4c5-43a7-bd36-57cdad1154d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:47.413268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904018f1-b4c5-43a7-bd36-57cdad1154d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:47.414731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904018f1-b4c5-43a7-bd36-57cdad1154d0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:47.434276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 391/777] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:47.434276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904018f1-b4c5-43a7-bd36-57cdad1154d0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:47.452534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 387/778] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:30:47.452534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820a2ab-e260-4d45-a9ac-af4958cd6428 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:47.540347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904018f1-b4c5-43a7-bd36-57cdad1154d0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:47.577941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590cef9-a169-40b2-8838-49df7ce7b8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:47.578998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590cef9-a169-40b2-8838-49df7ce7b8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.579629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590cef9-a169-40b2-8838-49df7ce7b8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.579997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590cef9-a169-40b2-8838-49df7ce7b8d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:47.580978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590cef9-a169-40b2-8838-49df7ce7b8d1 None admin] RBAC: Adding query filter params (interface=public, service_id=98b0e6831c8040ecb13e44d0294ec22e, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:47.581677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590cef9-a169-40b2-8838-49df7ce7b8d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:47.583421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590cef9-a169-40b2-8838-49df7ce7b8d1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:47.584014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820a2ab-e260-4d45-a9ac-af4958cd6428 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:47.596648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 392/779] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Aug 31 11:30:47 2022] GET /identity/v3/endpoints?service_id=98b0e6831c8040ecb13e44d0294ec22e&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:47.597492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590cef9-a169-40b2-8838-49df7ce7b8d1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:47.626228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:47.632425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968085b3-5fa5-48d2-a093-43c9f2831ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:47.633084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968085b3-5fa5-48d2-a093-43c9f2831ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.633710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968085b3-5fa5-48d2-a093-43c9f2831ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.633989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968085b3-5fa5-48d2-a093-43c9f2831ba4 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:47.635440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968085b3-5fa5-48d2-a093-43c9f2831ba4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:47.637381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968085b3-5fa5-48d2-a093-43c9f2831ba4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:47.643982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 388/780] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Aug 31 11:30:47 2022] GET /identity/v3/services => generated 772 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:47.644841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968085b3-5fa5-48d2-a093-43c9f2831ba4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:47.670059 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:47.758400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590cef9-a169-40b2-8838-49df7ce7b8d1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:47.799172 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87193f0-ef38-4609-ae24-acc1339c6e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:47.799954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87193f0-ef38-4609-ae24-acc1339c6e60 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.800338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87193f0-ef38-4609-ae24-acc1339c6e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.800678 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968085b3-5fa5-48d2-a093-43c9f2831ba4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:47.800885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87193f0-ef38-4609-ae24-acc1339c6e60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:47.801724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87193f0-ef38-4609-ae24-acc1339c6e60 None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=16216de2aa9c4a6388f94263444965df) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:47.802134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87193f0-ef38-4609-ae24-acc1339c6e60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:47.804554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87193f0-ef38-4609-ae24-acc1339c6e60 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:47.810829 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 393/781] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:47 2022] GET /identity/v3/role_assignments?user.id=16216de2aa9c4a6388f94263444965df&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:47.812921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87193f0-ef38-4609-ae24-acc1339c6e60 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:47.841247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80fe46-93aa-4dab-8e19-6982164195b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:47.842152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80fe46-93aa-4dab-8e19-6982164195b6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.842576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80fe46-93aa-4dab-8e19-6982164195b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.842987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80fe46-93aa-4dab-8e19-6982164195b6 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:47.843749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80fe46-93aa-4dab-8e19-6982164195b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:47.845356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80fe46-93aa-4dab-8e19-6982164195b6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:47.851603 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 389/782] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Aug 31 11:30:47 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:47.944793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87193f0-ef38-4609-ae24-acc1339c6e60 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:47.961279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:47.994361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624d523-8542-4d86-b7fb-3bbf6b913411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:47.995098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624d523-8542-4d86-b7fb-3bbf6b913411 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:47.995381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624d523-8542-4d86-b7fb-3bbf6b913411 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:47.995659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624d523-8542-4d86-b7fb-3bbf6b913411 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:48.009079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624d523-8542-4d86-b7fb-3bbf6b913411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-4624d523-8542-4d86-b7fb-3bbf6b913411 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:48.014472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:48.015432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:48.016279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 394/783] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:48.023307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80fe46-93aa-4dab-8e19-6982164195b6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:48.094997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:48.147542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80fe46-93aa-4dab-8e19-6982164195b6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:48.188189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea6055-8d9b-49ce-9b60-af0029da869b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:48.189073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6055-8d9b-49ce-9b60-af0029da869b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:48.189454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6055-8d9b-49ce-9b60-af0029da869b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:48.189823 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6055-8d9b-49ce-9b60-af0029da869b None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:48.190594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea6055-8d9b-49ce-9b60-af0029da869b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:48.190994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea6055-8d9b-49ce-9b60-af0029da869b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:48.192749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea6055-8d9b-49ce-9b60-af0029da869b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:48.202360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 390/784] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:48.207632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624d523-8542-4d86-b7fb-3bbf6b913411 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:48.323212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624d523-8542-4d86-b7fb-3bbf6b913411 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:48.368264 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056f78c-965e-453d-90f8-170b1386b805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:48.369522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056f78c-965e-453d-90f8-170b1386b805 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:48.370029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056f78c-965e-453d-90f8-170b1386b805 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:48.370499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056f78c-965e-453d-90f8-170b1386b805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:48.389600 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056f78c-965e-453d-90f8-170b1386b805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:48.391070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056f78c-965e-453d-90f8-170b1386b805 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5056f78c-965e-453d-90f8-170b1386b805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:48.406312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:48.407299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:48.408188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:48.408188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:48.408188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:48.408188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 395/785] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:48.413804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea6055-8d9b-49ce-9b60-af0029da869b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:48.579902 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea6055-8d9b-49ce-9b60-af0029da869b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:48.600971 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:48.626145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33bcad-82e9-40bb-b90f-1f05fe8eae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:48.626767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33bcad-82e9-40bb-b90f-1f05fe8eae50 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:48.627051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33bcad-82e9-40bb-b90f-1f05fe8eae50 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:48.627327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33bcad-82e9-40bb-b90f-1f05fe8eae50 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:48.627983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33bcad-82e9-40bb-b90f-1f05fe8eae50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:48.628288 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33bcad-82e9-40bb-b90f-1f05fe8eae50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:48.629721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33bcad-82e9-40bb-b90f-1f05fe8eae50 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:48.646585 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 391/786] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:48.653241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056f78c-965e-453d-90f8-170b1386b805 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:48.733856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:48.902977 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056f78c-965e-453d-90f8-170b1386b805 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:48.937784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c3be0-d10a-4a6f-9aa4-36465147250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:48.938412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c3be0-d10a-4a6f-9aa4-36465147250f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:48.938693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c3be0-d10a-4a6f-9aa4-36465147250f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:48.938965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c3be0-d10a-4a6f-9aa4-36465147250f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:48.947759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c3be0-d10a-4a6f-9aa4-36465147250f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:48.949279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c3be0-d10a-4a6f-9aa4-36465147250f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ac1c3be0-d10a-4a6f-9aa4-36465147250f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:48.960802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:48.961749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:48.962728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:48.962728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:48.962728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:48.962728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:48.962728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:48.962728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:48.962728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:48.962728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:48.962728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:48.962728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:48.962728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 396/787] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:48.968748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33bcad-82e9-40bb-b90f-1f05fe8eae50 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:49.062278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd29b73-2fc2-490c-aa60-281283ac5174 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:49.062625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd29b73-2fc2-490c-aa60-281283ac5174 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:49.062823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd29b73-2fc2-490c-aa60-281283ac5174 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:49.068996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 397/788] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:49.070615 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51953a9c-2a0f-4357-8c85-f778f5beb623 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:49.070899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51953a9c-2a0f-4357-8c85-f778f5beb623 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:49.071303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51953a9c-2a0f-4357-8c85-f778f5beb623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:49.093917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-51953a9c-2a0f-4357-8c85-f778f5beb623 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:49.095172 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-51953a9c-2a0f-4357-8c85-f778f5beb623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:49.115295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33bcad-82e9-40bb-b90f-1f05fe8eae50 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:49.169068 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59102646-5ad4-4781-9468-21a9835bb7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:49.169619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59102646-5ad4-4781-9468-21a9835bb7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:49.170010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59102646-5ad4-4781-9468-21a9835bb7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:49.170297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59102646-5ad4-4781-9468-21a9835bb7a7 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:49.171054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59102646-5ad4-4781-9468-21a9835bb7a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:49.171365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59102646-5ad4-4781-9468-21a9835bb7a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:49.172953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59102646-5ad4-4781-9468-21a9835bb7a7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:49.194378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 398/789] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:30:49.207167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 392/790] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Aug 31 11:30:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:49.217595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51953a9c-2a0f-4357-8c85-f778f5beb623 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:49.240844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:49.358728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51953a9c-2a0f-4357-8c85-f778f5beb623 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:49.409279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903ea4b-3c33-48b4-8c53-3a64ce5e008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:49.409911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903ea4b-3c33-48b4-8c53-3a64ce5e008b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:49.410265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903ea4b-3c33-48b4-8c53-3a64ce5e008b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:49.410551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903ea4b-3c33-48b4-8c53-3a64ce5e008b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:49.428087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903ea4b-3c33-48b4-8c53-3a64ce5e008b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:49.429481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903ea4b-3c33-48b4-8c53-3a64ce5e008b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2903ea4b-3c33-48b4-8c53-3a64ce5e008b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:49.446697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:49.448300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:49.450805 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:49.450805 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:49.450805 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:49.450805 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 399/791] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:49.461599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59102646-5ad4-4781-9468-21a9835bb7a7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:49.505193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903ea4b-3c33-48b4-8c53-3a64ce5e008b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:49.526394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:49.600964 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59102646-5ad4-4781-9468-21a9835bb7a7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:49.629272 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:49.671423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843db59-8979-408f-a581-de8ce1c4144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:49.672075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843db59-8979-408f-a581-de8ce1c4144b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:49.672302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843db59-8979-408f-a581-de8ce1c4144b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:49.672595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843db59-8979-408f-a581-de8ce1c4144b None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:49.673247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843db59-8979-408f-a581-de8ce1c4144b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:49.673547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843db59-8979-408f-a581-de8ce1c4144b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:49.683136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843db59-8979-408f-a581-de8ce1c4144b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:49.707460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903ea4b-3c33-48b4-8c53-3a64ce5e008b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:49.707781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 393/792] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:49.717009 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843db59-8979-408f-a581-de8ce1c4144b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:49.785222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569a608-5cbb-42df-8410-aa9e2dce7b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:49.786151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a608-5cbb-42df-8410-aa9e2dce7b7a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:49.792409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a608-5cbb-42df-8410-aa9e2dce7b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:49.792902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a608-5cbb-42df-8410-aa9e2dce7b7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:49.814430 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569a608-5cbb-42df-8410-aa9e2dce7b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e569a608-5cbb-42df-8410-aa9e2dce7b7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:49.820266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:49.821440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:49.823806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:49.826249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 400/793] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:49.831856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569a608-5cbb-42df-8410-aa9e2dce7b7a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:49.910063 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:49.921183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843db59-8979-408f-a581-de8ce1c4144b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:49.998680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae756f56-fe05-4d26-b5b3-cfd86148fc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:49.999665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae756f56-fe05-4d26-b5b3-cfd86148fc47 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.000166 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae756f56-fe05-4d26-b5b3-cfd86148fc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.000680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae756f56-fe05-4d26-b5b3-cfd86148fc47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.014429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae756f56-fe05-4d26-b5b3-cfd86148fc47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:50.016890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae756f56-fe05-4d26-b5b3-cfd86148fc47 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:50.031609 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569a608-5cbb-42df-8410-aa9e2dce7b7a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-ae756f56-fe05-4d26-b5b3-cfd86148fc47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:50.039326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:50.040231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:50.042318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:50.042318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:50.042318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:50.042318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:50.042318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:50.042318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:50.042318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:50.042318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:50.058217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 394/794] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:50.064831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae756f56-fe05-4d26-b5b3-cfd86148fc47 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:50.090432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:50.104453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06351bf-520a-48af-96d5-d31cdd3bd755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:50.105108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06351bf-520a-48af-96d5-d31cdd3bd755 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.105393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06351bf-520a-48af-96d5-d31cdd3bd755 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.105671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06351bf-520a-48af-96d5-d31cdd3bd755 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.106344 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06351bf-520a-48af-96d5-d31cdd3bd755 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:50.106653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06351bf-520a-48af-96d5-d31cdd3bd755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:50.108140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06351bf-520a-48af-96d5-d31cdd3bd755 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:50.117700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 401/795] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:50.120062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed2a46-38d6-4a06-b980-6c015dbc223b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.120621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed2a46-38d6-4a06-b980-6c015dbc223b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.124384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed2a46-38d6-4a06-b980-6c015dbc223b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.130081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 402/796] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:50.130081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed2a46-38d6-4a06-b980-6c015dbc223b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:50.204405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae756f56-fe05-4d26-b5b3-cfd86148fc47 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:50.245372 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed2a46-38d6-4a06-b980-6c015dbc223b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:50.247959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a17be-5dbb-4a65-ba7f-b5d513b36ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:50.248611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a17be-5dbb-4a65-ba7f-b5d513b36ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.248951 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a17be-5dbb-4a65-ba7f-b5d513b36ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.249213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a17be-5dbb-4a65-ba7f-b5d513b36ec0 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.250223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a17be-5dbb-4a65-ba7f-b5d513b36ec0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:50.250343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a17be-5dbb-4a65-ba7f-b5d513b36ec0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:50.251856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a17be-5dbb-4a65-ba7f-b5d513b36ec0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:50.272117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 395/797] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:50.273726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8bb90-dc5d-4758-b010-cdbbd84693ce None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.274022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8bb90-dc5d-4758-b010-cdbbd84693ce None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.274298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8bb90-dc5d-4758-b010-cdbbd84693ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.284358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4eedd-0241-4d6d-9855-92de6c4a2e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:50.285243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4eedd-0241-4d6d-9855-92de6c4a2e66 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.285675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4eedd-0241-4d6d-9855-92de6c4a2e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.286087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4eedd-0241-4d6d-9855-92de6c4a2e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.294453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f5a8bb90-dc5d-4758-b010-cdbbd84693ce None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:50.295674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f5a8bb90-dc5d-4758-b010-cdbbd84693ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:50.297016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4eedd-0241-4d6d-9855-92de6c4a2e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:50.298597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4eedd-0241-4d6d-9855-92de6c4a2e66 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:50.310453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8af4eedd-0241-4d6d-9855-92de6c4a2e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:50.314087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:50.315029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:50.315954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:50.315954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:50.315954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:50.315954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 403/798] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:50.316884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140b69-14d9-4993-9b04-935406deb1d2 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.316986 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140b69-14d9-4993-9b04-935406deb1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.317378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140b69-14d9-4993-9b04-935406deb1d2 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.320618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 404/799] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:50.320618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb140b69-14d9-4993-9b04-935406deb1d2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:50.366001 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:50.378733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 396/800] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:30:50.379688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8bb90-dc5d-4758-b010-cdbbd84693ce None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:50.436138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb140b69-14d9-4993-9b04-935406deb1d2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:50.476301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad47059-28b9-4694-a11a-1795c9238dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:50.476980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad47059-28b9-4694-a11a-1795c9238dcf None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.477280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad47059-28b9-4694-a11a-1795c9238dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.477604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad47059-28b9-4694-a11a-1795c9238dcf None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/3384c24dd081414bbf0a2e3d9f71126b/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.480089 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad47059-28b9-4694-a11a-1795c9238dcf None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=3384c24dd081414bbf0a2e3d9f71126b, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:50.481545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad47059-28b9-4694-a11a-1795c9238dcf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:50.495548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8bb90-dc5d-4758-b010-cdbbd84693ce None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:50.504094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 405/801] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:30:50 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/3384c24dd081414bbf0a2e3d9f71126b/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:30:50.504094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d09cb5-0a7d-4f06-9c19-1b4df7ea940b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.504094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d09cb5-0a7d-4f06-9c19-1b4df7ea940b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.504094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d09cb5-0a7d-4f06-9c19-1b4df7ea940b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.519341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-75d09cb5-0a7d-4f06-9c19-1b4df7ea940b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:50.520752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-75d09cb5-0a7d-4f06-9c19-1b4df7ea940b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:50.547152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd4759-f51e-49e7-bf1c-cf55ff9062b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:50.548279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd4759-f51e-49e7-bf1c-cf55ff9062b5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.548833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd4759-f51e-49e7-bf1c-cf55ff9062b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.549282 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd4759-f51e-49e7-bf1c-cf55ff9062b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.550181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd4759-f51e-49e7-bf1c-cf55ff9062b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:50.550710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd4759-f51e-49e7-bf1c-cf55ff9062b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:50.552541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd4759-f51e-49e7-bf1c-cf55ff9062b5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:50.568592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 397/802] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:50.570267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c3b1c-0787-4674-9e43-68ce82cf2952 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.570744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c3b1c-0787-4674-9e43-68ce82cf2952 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.571193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c3b1c-0787-4674-9e43-68ce82cf2952 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.574116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 398/803] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:50.575165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c3b1c-0787-4674-9e43-68ce82cf2952 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:50.588767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:50.610901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 406/804] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:30:50.612240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd4312-e8e4-4649-a6c6-6d26027932d0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.612545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd4312-e8e4-4649-a6c6-6d26027932d0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.612823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd4312-e8e4-4649-a6c6-6d26027932d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.632328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f7cd4312-e8e4-4649-a6c6-6d26027932d0 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:50.633723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f7cd4312-e8e4-4649-a6c6-6d26027932d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:50.663039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:50.711386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c3b1c-0787-4674-9e43-68ce82cf2952 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:50.735392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 407/805] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:30:50.736257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd4312-e8e4-4649-a6c6-6d26027932d0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:50.751327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acff87-8ad4-4c04-a02b-06d738d1136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:50.751963 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acff87-8ad4-4c04-a02b-06d738d1136f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.752244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acff87-8ad4-4c04-a02b-06d738d1136f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.752549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acff87-8ad4-4c04-a02b-06d738d1136f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.761658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50acff87-8ad4-4c04-a02b-06d738d1136f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:50.763042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50acff87-8ad4-4c04-a02b-06d738d1136f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-50acff87-8ad4-4c04-a02b-06d738d1136f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:50.776093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:50.777140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:50.778097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:50.778097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:50.778097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:50.778097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:50.778097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:50.778097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:50.778097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:50.778097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:50.778097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:30:50.778097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:50.778097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 399/806] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Aug 31 11:30:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:50.783761 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acff87-8ad4-4c04-a02b-06d738d1136f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:50.861779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:50.870057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd4312-e8e4-4649-a6c6-6d26027932d0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:50.900511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acff87-8ad4-4c04-a02b-06d738d1136f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:50.923915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b567d28-16ee-49e8-9d5a-e610e8d2f7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:50.925951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b567d28-16ee-49e8-9d5a-e610e8d2f7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.926231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b567d28-16ee-49e8-9d5a-e610e8d2f7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.926501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b567d28-16ee-49e8-9d5a-e610e8d2f7a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.948268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b567d28-16ee-49e8-9d5a-e610e8d2f7a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3b567d28-16ee-49e8-9d5a-e610e8d2f7a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:50.956553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:50.957446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:50.958273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 408/807] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:50.963939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc0a3c-632c-4a39-9b0e-1c4bb17c3ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:50.964595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc0a3c-632c-4a39-9b0e-1c4bb17c3ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:50.964936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc0a3c-632c-4a39-9b0e-1c4bb17c3ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:50.965214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc0a3c-632c-4a39-9b0e-1c4bb17c3ba6 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:50.966600 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b567d28-16ee-49e8-9d5a-e610e8d2f7a9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:50.966956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc0a3c-632c-4a39-9b0e-1c4bb17c3ba6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:50.967270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc0a3c-632c-4a39-9b0e-1c4bb17c3ba6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:50.970625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc0a3c-632c-4a39-9b0e-1c4bb17c3ba6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:50.996348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:51.002291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 400/808] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Aug 31 11:30:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:51.003677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc0a3c-632c-4a39-9b0e-1c4bb17c3ba6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:51.084463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b567d28-16ee-49e8-9d5a-e610e8d2f7a9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:51.118865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc0a3c-632c-4a39-9b0e-1c4bb17c3ba6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:51.120787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58387f-d30f-48e3-8bc5-9c4aef6617d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:51.121400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58387f-d30f-48e3-8bc5-9c4aef6617d3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:51.121698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58387f-d30f-48e3-8bc5-9c4aef6617d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:51.121943 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58387f-d30f-48e3-8bc5-9c4aef6617d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:51.122614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58387f-d30f-48e3-8bc5-9c4aef6617d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:51.122918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58387f-d30f-48e3-8bc5-9c4aef6617d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:51.124272 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58387f-d30f-48e3-8bc5-9c4aef6617d3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:51.132468 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 409/809] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:51.133292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58387f-d30f-48e3-8bc5-9c4aef6617d3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:51.159443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:51.172393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee315b-89a5-4e23-a6a9-ecdb85aa28a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:51.173116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee315b-89a5-4e23-a6a9-ecdb85aa28a9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:51.173464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee315b-89a5-4e23-a6a9-ecdb85aa28a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:51.173814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee315b-89a5-4e23-a6a9-ecdb85aa28a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:51.232732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:51.255770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58387f-d30f-48e3-8bc5-9c4aef6617d3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:51.295025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb479ca-3f7b-48d8-bfb1-47ba08ad6b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:51.295670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb479ca-3f7b-48d8-bfb1-47ba08ad6b89 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:51.295955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb479ca-3f7b-48d8-bfb1-47ba08ad6b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:51.296233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb479ca-3f7b-48d8-bfb1-47ba08ad6b89 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:51.296856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb479ca-3f7b-48d8-bfb1-47ba08ad6b89 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:51.298231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb479ca-3f7b-48d8-bfb1-47ba08ad6b89 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7eb479ca-3f7b-48d8-bfb1-47ba08ad6b89 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:51.307808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:51.309212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:51.310043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 410/810] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Aug 31 11:30:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:51.310912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb479ca-3f7b-48d8-bfb1-47ba08ad6b89 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:51.367142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee315b-89a5-4e23-a6a9-ecdb85aa28a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:51.368536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee315b-89a5-4e23-a6a9-ecdb85aa28a9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:51.417174 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb479ca-3f7b-48d8-bfb1-47ba08ad6b89 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:51.453949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416b4f3-f21a-4a5a-82ee-8f93c02aa93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:51.454571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416b4f3-f21a-4a5a-82ee-8f93c02aa93b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:51.454852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416b4f3-f21a-4a5a-82ee-8f93c02aa93b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:51.455131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416b4f3-f21a-4a5a-82ee-8f93c02aa93b None admin] PATH_INFO: `/v3/services/network` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:51.455707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416b4f3-f21a-4a5a-82ee-8f93c02aa93b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:51.457092 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416b4f3-f21a-4a5a-82ee-8f93c02aa93b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6416b4f3-f21a-4a5a-82ee-8f93c02aa93b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:51.466367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:51.467324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:51.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 411/811] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:51 2022] GET /identity/v3/services/network => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:51.468985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416b4f3-f21a-4a5a-82ee-8f93c02aa93b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:51.481304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 401/812] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 11:30:51.481304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee315b-89a5-4e23-a6a9-ecdb85aa28a9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:51.484025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:51.506991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:51.585192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416b4f3-f21a-4a5a-82ee-8f93c02aa93b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:51.585394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee315b-89a5-4e23-a6a9-ecdb85aa28a9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:51.621034 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a8d1d-c77c-403f-a831-308dd371563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:51.621558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a8d1d-c77c-403f-a831-308dd371563f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:51.621865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a8d1d-c77c-403f-a831-308dd371563f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:51.622100 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a8d1d-c77c-403f-a831-308dd371563f None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:51.622719 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a8d1d-c77c-403f-a831-308dd371563f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:51.623018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a8d1d-c77c-403f-a831-308dd371563f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:51.623526 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01b0d5-b830-44c7-bad2-09cc250b9452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:51.624150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01b0d5-b830-44c7-bad2-09cc250b9452 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:51.624366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a8d1d-c77c-403f-a831-308dd371563f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:51.624465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01b0d5-b830-44c7-bad2-09cc250b9452 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:51.624723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01b0d5-b830-44c7-bad2-09cc250b9452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:51.630093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 402/813] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Aug 31 11:30:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:51.631183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a8d1d-c77c-403f-a831-308dd371563f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:51.636207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01b0d5-b830-44c7-bad2-09cc250b9452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:51.637664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01b0d5-b830-44c7-bad2-09cc250b9452 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-4e01b0d5-b830-44c7-bad2-09cc250b9452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:51.652884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:51.653875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:51.654809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:51.654809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:51.654809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:51.654809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 412/814] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:51.654809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01b0d5-b830-44c7-bad2-09cc250b9452 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:51.727459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:51.743263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:51.754701 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a8d1d-c77c-403f-a831-308dd371563f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:51.773006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01b0d5-b830-44c7-bad2-09cc250b9452 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:51.793814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1eeb6-0506-45ae-a3f4-32e825cd7155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:51.794515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1eeb6-0506-45ae-a3f4-32e825cd7155 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:51.794860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1eeb6-0506-45ae-a3f4-32e825cd7155 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:51.795200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1eeb6-0506-45ae-a3f4-32e825cd7155 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:51.795900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1eeb6-0506-45ae-a3f4-32e825cd7155 None admin] RBAC: Adding query filter params (name=network) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:51.796539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1eeb6-0506-45ae-a3f4-32e825cd7155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:51.797663 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1eeb6-0506-45ae-a3f4-32e825cd7155 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:51.802153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 403/815] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:30:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:51.802921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1eeb6-0506-45ae-a3f4-32e825cd7155 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:51.815219 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fd703-1cdd-411a-ba92-b61da1ddbcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:51.815840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd703-1cdd-411a-ba92-b61da1ddbcf3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:51.816117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd703-1cdd-411a-ba92-b61da1ddbcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:51.816387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd703-1cdd-411a-ba92-b61da1ddbcf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:51.828029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fd703-1cdd-411a-ba92-b61da1ddbcf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:51.829476 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fd703-1cdd-411a-ba92-b61da1ddbcf3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-fd9fd703-1cdd-411a-ba92-b61da1ddbcf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:51.844205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:51.845200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:51.846097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:51.846097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:51.846097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:51.846097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 413/816] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:51.846097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fd703-1cdd-411a-ba92-b61da1ddbcf3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:51.911647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1eeb6-0506-45ae-a3f4-32e825cd7155 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:51.951187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fd703-1cdd-411a-ba92-b61da1ddbcf3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:51.955357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ac985-d82d-485b-92d4-29b496598c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:51.955972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ac985-d82d-485b-92d4-29b496598c2b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:51.956255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ac985-d82d-485b-92d4-29b496598c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:51.956558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ac985-d82d-485b-92d4-29b496598c2b None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:51.957195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ac985-d82d-485b-92d4-29b496598c2b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:51.957501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ac985-d82d-485b-92d4-29b496598c2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:51.959277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ac985-d82d-485b-92d4-29b496598c2b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:51.964650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:51.970807 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 404/817] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Aug 31 11:30:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:51.970807 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ac985-d82d-485b-92d4-29b496598c2b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:51.990392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:51.991734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ada216-3726-49a2-a4ba-f91e25232b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:51.992331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada216-3726-49a2-a4ba-f91e25232b83 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:51.992702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada216-3726-49a2-a4ba-f91e25232b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:51.992975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada216-3726-49a2-a4ba-f91e25232b83 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:51.993643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ada216-3726-49a2-a4ba-f91e25232b83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:51.994002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ada216-3726-49a2-a4ba-f91e25232b83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:51.995361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ada216-3726-49a2-a4ba-f91e25232b83 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:52.013078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 414/818] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:52.013078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ada216-3726-49a2-a4ba-f91e25232b83 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:52.091001 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ac985-d82d-485b-92d4-29b496598c2b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:52.120176 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ada216-3726-49a2-a4ba-f91e25232b83 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:52.127155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6c40b-d313-431e-acb2-7e15f64561ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:52.127767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6c40b-d313-431e-acb2-7e15f64561ca None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:52.128050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6c40b-d313-431e-acb2-7e15f64561ca None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:52.128322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6c40b-d313-431e-acb2-7e15f64561ca None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:52.129412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6c40b-d313-431e-acb2-7e15f64561ca None admin] RBAC: Adding query filter params (type=network) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:52.129718 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6c40b-d313-431e-acb2-7e15f64561ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:52.131473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6c40b-d313-431e-acb2-7e15f64561ca None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:52.137837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 405/819] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:30:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:52.138534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6c40b-d313-431e-acb2-7e15f64561ca None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:52.162863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f9a08-990e-45aa-a661-3bd63cc51796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:52.163393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f9a08-990e-45aa-a661-3bd63cc51796 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:52.163829 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f9a08-990e-45aa-a661-3bd63cc51796 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:52.164101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f9a08-990e-45aa-a661-3bd63cc51796 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:52.164778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f9a08-990e-45aa-a661-3bd63cc51796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:52.165109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f9a08-990e-45aa-a661-3bd63cc51796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:52.166473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f9a08-990e-45aa-a661-3bd63cc51796 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:52.182648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 415/820] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:52.182648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f9a08-990e-45aa-a661-3bd63cc51796 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:52.216142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:52.228855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:52.246849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6c40b-d313-431e-acb2-7e15f64561ca None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:52.281434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab3ad0-b987-4883-9cf1-c48bf73db1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:52.282163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab3ad0-b987-4883-9cf1-c48bf73db1dd None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:52.282594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab3ad0-b987-4883-9cf1-c48bf73db1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:52.282877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab3ad0-b987-4883-9cf1-c48bf73db1dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:52.283573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab3ad0-b987-4883-9cf1-c48bf73db1dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:52.285063 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab3ad0-b987-4883-9cf1-c48bf73db1dd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:52.295285 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f9a08-990e-45aa-a661-3bd63cc51796 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:52.311225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 406/821] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:30:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:30:52.312039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab3ad0-b987-4883-9cf1-c48bf73db1dd None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:52.370083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8778b-e058-4803-a2dc-82d1ea98e52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:52.371376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8778b-e058-4803-a2dc-82d1ea98e52d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:52.372292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8778b-e058-4803-a2dc-82d1ea98e52d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:52.372855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8778b-e058-4803-a2dc-82d1ea98e52d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:52.384702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8778b-e058-4803-a2dc-82d1ea98e52d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:52.387750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8778b-e058-4803-a2dc-82d1ea98e52d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1be8778b-e058-4803-a2dc-82d1ea98e52d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:52.405358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:52.406360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:52.407297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:52.407297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:52.407297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:52.407297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:52.407297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:52.407297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:52.407297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:52.407297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:52.407297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:30:52.407297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:52.414694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 416/822] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Aug 31 11:30:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:52.414694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8778b-e058-4803-a2dc-82d1ea98e52d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:52.440390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab3ad0-b987-4883-9cf1-c48bf73db1dd None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:52.480148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56ee1a-7fb1-44ac-b730-6ef1cc6dcfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:52.480804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56ee1a-7fb1-44ac-b730-6ef1cc6dcfb8 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:52.481136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56ee1a-7fb1-44ac-b730-6ef1cc6dcfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:52.481412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56ee1a-7fb1-44ac-b730-6ef1cc6dcfb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:52.482261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56ee1a-7fb1-44ac-b730-6ef1cc6dcfb8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:52.483797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56ee1a-7fb1-44ac-b730-6ef1cc6dcfb8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:52.493916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:52.508524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 407/823] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:30:52 2022] POST /identity/v3/endpoints => generated 337 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:30:52.510028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56ee1a-7fb1-44ac-b730-6ef1cc6dcfb8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:52.513508 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:52.532333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8778b-e058-4803-a2dc-82d1ea98e52d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:52.584449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c74de-6061-413a-8529-c30f62099062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:52.586432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c74de-6061-413a-8529-c30f62099062 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:52.586707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c74de-6061-413a-8529-c30f62099062 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:52.586976 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c74de-6061-413a-8529-c30f62099062 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:52.600344 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c74de-6061-413a-8529-c30f62099062 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:52.603069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c74de-6061-413a-8529-c30f62099062 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8b8c74de-6061-413a-8529-c30f62099062 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:52.619987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:52.621044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:52.621948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:52.621948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:52.621948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:52.621948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:52.621948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:52.621948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:52.621948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:52.621948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:52.624709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 417/824] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:52.630745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c74de-6061-413a-8529-c30f62099062 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:52.654779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56ee1a-7fb1-44ac-b730-6ef1cc6dcfb8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:52.705911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada97f9-437b-434f-86f7-e48852ec4126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:52.707232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada97f9-437b-434f-86f7-e48852ec4126 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:52.707651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada97f9-437b-434f-86f7-e48852ec4126 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:52.708480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada97f9-437b-434f-86f7-e48852ec4126 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:52.709703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada97f9-437b-434f-86f7-e48852ec4126 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:52.710156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada97f9-437b-434f-86f7-e48852ec4126 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:52.713019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada97f9-437b-434f-86f7-e48852ec4126 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:52.739033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 408/825] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Aug 31 11:30:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:52.740752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe68c1c-a91f-490f-99db-d729e6ad7ed2 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:52.741204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe68c1c-a91f-490f-99db-d729e6ad7ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:52.741482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe68c1c-a91f-490f-99db-d729e6ad7ed2 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:52.743853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 409/826] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:52.748655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe68c1c-a91f-490f-99db-d729e6ad7ed2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:52.780221 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c74de-6061-413a-8529-c30f62099062 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:52.826868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:52.827565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6712b6-de0a-441f-a26a-e7c65a7233c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:52.828188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6712b6-de0a-441f-a26a-e7c65a7233c9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:52.828472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6712b6-de0a-441f-a26a-e7c65a7233c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:52.828781 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6712b6-de0a-441f-a26a-e7c65a7233c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:52.829606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6712b6-de0a-441f-a26a-e7c65a7233c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:52.829995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6712b6-de0a-441f-a26a-e7c65a7233c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:52.831718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6712b6-de0a-441f-a26a-e7c65a7233c9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:52.845814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 418/827] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:52.846940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4fd9-9b3b-4555-8a19-662c0840e101 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:52.847224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4fd9-9b3b-4555-8a19-662c0840e101 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:52.847497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4fd9-9b3b-4555-8a19-662c0840e101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:52.856182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:52.869311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-274c4fd9-9b3b-4555-8a19-662c0840e101 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:52.870924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-274c4fd9-9b3b-4555-8a19-662c0840e101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:52.880203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe68c1c-a91f-490f-99db-d729e6ad7ed2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:52.916801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4e43b-73f2-4212-b1b5-d8fcb050b906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:52.917434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4e43b-73f2-4212-b1b5-d8fcb050b906 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:52.917716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4e43b-73f2-4212-b1b5-d8fcb050b906 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:52.917991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4e43b-73f2-4212-b1b5-d8fcb050b906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:52.928608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4e43b-73f2-4212-b1b5-d8fcb050b906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:52.930107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4e43b-73f2-4212-b1b5-d8fcb050b906 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:52.946437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 419/828] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-41b4e43b-73f2-4212-b1b5-d8fcb050b906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:52.946850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:52.948432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:52.949338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:52.949338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:52.949338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:52.949515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c4fd9-9b3b-4555-8a19-662c0840e101 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:52.955193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 410/829] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:52.956450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4e43b-73f2-4212-b1b5-d8fcb050b906 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:53.073487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c4fd9-9b3b-4555-8a19-662c0840e101 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:53.080432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4e43b-73f2-4212-b1b5-d8fcb050b906 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:53.083564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:53.113767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c27270-8bd2-4a42-b067-6981b227d337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:53.114797 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c27270-8bd2-4a42-b067-6981b227d337 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:53.115328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c27270-8bd2-4a42-b067-6981b227d337 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:53.115824 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c27270-8bd2-4a42-b067-6981b227d337 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:53.116781 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c27270-8bd2-4a42-b067-6981b227d337 None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=f4048316ea784c389eb249b4dd915e0e) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:53.117343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c27270-8bd2-4a42-b067-6981b227d337 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:53.119214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c27270-8bd2-4a42-b067-6981b227d337 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:53.121414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:53.124053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925e849-4ffe-425c-8972-f65894b3f9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:53.125107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925e849-4ffe-425c-8972-f65894b3f9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:53.125337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925e849-4ffe-425c-8972-f65894b3f9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:53.125615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925e849-4ffe-425c-8972-f65894b3f9b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:53.126264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925e849-4ffe-425c-8972-f65894b3f9b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:53.126567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925e849-4ffe-425c-8972-f65894b3f9b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:53.128281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 420/830] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:52 2022] GET /identity/v3/role_assignments?user.id=f4048316ea784c389eb249b4dd915e0e&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:53.128482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925e849-4ffe-425c-8972-f65894b3f9b2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:53.142814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 411/831] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:53.148961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c27270-8bd2-4a42-b067-6981b227d337 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:53.233034 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925e849-4ffe-425c-8972-f65894b3f9b2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:53.280394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c27270-8bd2-4a42-b067-6981b227d337 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:53.334654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81b73a-f8cf-4ac7-bc47-271f8d1b1fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:53.335404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b73a-f8cf-4ac7-bc47-271f8d1b1fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:53.335685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b73a-f8cf-4ac7-bc47-271f8d1b1fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:53.335955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b73a-f8cf-4ac7-bc47-271f8d1b1fc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:53.346554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81b73a-f8cf-4ac7-bc47-271f8d1b1fc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:53.347988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81b73a-f8cf-4ac7-bc47-271f8d1b1fc4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:53.349753 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925e849-4ffe-425c-8972-f65894b3f9b2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0c81b73a-f8cf-4ac7-bc47-271f8d1b1fc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:53.362869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:53.363799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:53.364717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:53.364717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:53.364717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:53.364717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:53.364717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:53.364717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:53.364717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:53.364717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:53.364717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 421/832] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:53.371067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81b73a-f8cf-4ac7-bc47-271f8d1b1fc4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:53.390613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3211d-fa7e-47fd-a77a-f4d81ff097f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:53.391311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3211d-fa7e-47fd-a77a-f4d81ff097f3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:53.391595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3211d-fa7e-47fd-a77a-f4d81ff097f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:53.391870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3211d-fa7e-47fd-a77a-f4d81ff097f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:53.404946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3211d-fa7e-47fd-a77a-f4d81ff097f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-48d3211d-fa7e-47fd-a77a-f4d81ff097f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:53.411968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:53.412937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:53.413776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 412/833] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:53.420024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3211d-fa7e-47fd-a77a-f4d81ff097f3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:53.450360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:53.464780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:53.525950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81b73a-f8cf-4ac7-bc47-271f8d1b1fc4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:53.546189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3211d-fa7e-47fd-a77a-f4d81ff097f3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:53.563762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946c951-06f8-49b4-ba54-63dc013c1a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:53.564306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c951-06f8-49b4-ba54-63dc013c1a6f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:53.564623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c951-06f8-49b4-ba54-63dc013c1a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:53.564896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c951-06f8-49b4-ba54-63dc013c1a6f None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:53.565581 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946c951-06f8-49b4-ba54-63dc013c1a6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:53.565880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946c951-06f8-49b4-ba54-63dc013c1a6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:53.567548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946c951-06f8-49b4-ba54-63dc013c1a6f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:53.581346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 422/834] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:53.586798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946c951-06f8-49b4-ba54-63dc013c1a6f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:53.595993 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8f023-8d30-47b7-a59b-02b51d03f45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:53.596952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8f023-8d30-47b7-a59b-02b51d03f45b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:53.597440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8f023-8d30-47b7-a59b-02b51d03f45b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:53.597917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8f023-8d30-47b7-a59b-02b51d03f45b None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:53.598827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8f023-8d30-47b7-a59b-02b51d03f45b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:53.599332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8f023-8d30-47b7-a59b-02b51d03f45b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:53.600935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8f023-8d30-47b7-a59b-02b51d03f45b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:53.612037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 413/835] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:53.617711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8f023-8d30-47b7-a59b-02b51d03f45b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:53.719739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946c951-06f8-49b4-ba54-63dc013c1a6f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:53.724230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:53.739468 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:53.744444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8f023-8d30-47b7-a59b-02b51d03f45b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:53.764618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122017c7-64bd-4451-84cc-c78e6583f5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:53.765862 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122017c7-64bd-4451-84cc-c78e6583f5cc None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:53.766348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122017c7-64bd-4451-84cc-c78e6583f5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:53.766856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122017c7-64bd-4451-84cc-c78e6583f5cc None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/16216de2aa9c4a6388f94263444965df/roles/babbae9ac71c4c15817721af5d270a80` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:53.783150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6e336-0fd6-4208-9f9f-85336db17254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:53.783732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6e336-0fd6-4208-9f9f-85336db17254 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:53.784019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6e336-0fd6-4208-9f9f-85336db17254 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:53.784331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6e336-0fd6-4208-9f9f-85336db17254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:53.785969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122017c7-64bd-4451-84cc-c78e6583f5cc None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=16216de2aa9c4a6388f94263444965df, role_id=babbae9ac71c4c15817721af5d270a80)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:53.788066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122017c7-64bd-4451-84cc-c78e6583f5cc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:53.796994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6e336-0fd6-4208-9f9f-85336db17254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:53.798628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6e336-0fd6-4208-9f9f-85336db17254 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:53.813050 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 423/836] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:30:53 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/16216de2aa9c4a6388f94263444965df/roles/babbae9ac71c4c15817721af5d270a80 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-1be6e336-0fd6-4208-9f9f-85336db17254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:53.823468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:53.824503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:53.825822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:53.825822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:53.825822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:53.825822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 414/837] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:53.831142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122017c7-64bd-4451-84cc-c78e6583f5cc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:53.947639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122017c7-64bd-4451-84cc-c78e6583f5cc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:53.986126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d68fb-755e-43b4-89bc-cceb6f6b2773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:53.986736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d68fb-755e-43b4-89bc-cceb6f6b2773 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:53.987011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d68fb-755e-43b4-89bc-cceb6f6b2773 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:53.987278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d68fb-755e-43b4-89bc-cceb6f6b2773 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:53.987923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d68fb-755e-43b4-89bc-cceb6f6b2773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:53.988222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d68fb-755e-43b4-89bc-cceb6f6b2773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:53.989608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d68fb-755e-43b4-89bc-cceb6f6b2773 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:54.003437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 424/838] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:54.009086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6e336-0fd6-4208-9f9f-85336db17254 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:54.118011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6e336-0fd6-4208-9f9f-85336db17254 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:54.154766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87997b59-354c-49fd-a750-b5d61e97cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:54.155389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87997b59-354c-49fd-a750-b5d61e97cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:54.155713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87997b59-354c-49fd-a750-b5d61e97cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:54.155989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87997b59-354c-49fd-a750-b5d61e97cdc6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:54.164185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87997b59-354c-49fd-a750-b5d61e97cdc6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:54.165723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87997b59-354c-49fd-a750-b5d61e97cdc6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:54.167721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-87997b59-354c-49fd-a750-b5d61e97cdc6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:54.179080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:54.180021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:54.181180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:54.181180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:54.181180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:54.181180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:54.181180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:54.181180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:54.181180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:54.181180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:54.181180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 11:30:54.181180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:54.181180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 415/839] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:30:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:54.189594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d68fb-755e-43b4-89bc-cceb6f6b2773 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:54.220788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:54.296734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d68fb-755e-43b4-89bc-cceb6f6b2773 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:54.333209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454d81c-f677-467f-b6a8-17bdfee9c492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:54.334194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454d81c-f677-467f-b6a8-17bdfee9c492 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:54.334683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454d81c-f677-467f-b6a8-17bdfee9c492 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:54.335615 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454d81c-f677-467f-b6a8-17bdfee9c492 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:54.336579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454d81c-f677-467f-b6a8-17bdfee9c492 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:54.337111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454d81c-f677-467f-b6a8-17bdfee9c492 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:54.338916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454d81c-f677-467f-b6a8-17bdfee9c492 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:54.364597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 425/840] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Aug 31 11:30:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:54.366721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87997b59-354c-49fd-a750-b5d61e97cdc6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:54.421720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7205f0-e030-452b-a7f2-8c2ca922fc37 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:54.421916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7205f0-e030-452b-a7f2-8c2ca922fc37 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:54.422191 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7205f0-e030-452b-a7f2-8c2ca922fc37 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:54.423773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 426/841] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:30:54.428563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5cec5-a58b-46f3-9038-ab8e04fe6fee None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:54.428853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5cec5-a58b-46f3-9038-ab8e04fe6fee None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:54.429130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5cec5-a58b-46f3-9038-ab8e04fe6fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:54.450383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8ca5cec5-a58b-46f3-9038-ab8e04fe6fee None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:54.451629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8ca5cec5-a58b-46f3-9038-ab8e04fe6fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:54.488196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87997b59-354c-49fd-a750-b5d61e97cdc6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:54.533462 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1198af-68f5-44f3-9d85-5ed8e73bdecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:54.534062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1198af-68f5-44f3-9d85-5ed8e73bdecb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:54.534339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1198af-68f5-44f3-9d85-5ed8e73bdecb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:54.534612 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1198af-68f5-44f3-9d85-5ed8e73bdecb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:54.536820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 427/842] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:30:54.611154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:54.614860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1198af-68f5-44f3-9d85-5ed8e73bdecb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:54.616362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1198af-68f5-44f3-9d85-5ed8e73bdecb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:54.726523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 416/843] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:54 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:30:54.732458 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5cec5-a58b-46f3-9038-ab8e04fe6fee None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:54.751274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:54.840268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5cec5-a58b-46f3-9038-ab8e04fe6fee None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:54.842142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1198af-68f5-44f3-9d85-5ed8e73bdecb None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:54.878749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a89bfb-ecd0-4c45-a5b2-ea3b171de391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:54.879736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a89bfb-ecd0-4c45-a5b2-ea3b171de391 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:54.880219 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a89bfb-ecd0-4c45-a5b2-ea3b171de391 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:54.880750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a89bfb-ecd0-4c45-a5b2-ea3b171de391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:54.893239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a89bfb-ecd0-4c45-a5b2-ea3b171de391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:54.895010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a89bfb-ecd0-4c45-a5b2-ea3b171de391 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-44a89bfb-ecd0-4c45-a5b2-ea3b171de391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:54.910678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:54.911688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:54.912603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:54.912603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:54.912603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:54.914674 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 428/844] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:54.921298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a89bfb-ecd0-4c45-a5b2-ea3b171de391 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:54.966108 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:54.972512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1198af-68f5-44f3-9d85-5ed8e73bdecb None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:55.011377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:55.019750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd7a1e-9a99-4a93-8a3a-8d723622280d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:55.020765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd7a1e-9a99-4a93-8a3a-8d723622280d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.021259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd7a1e-9a99-4a93-8a3a-8d723622280d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.021739 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd7a1e-9a99-4a93-8a3a-8d723622280d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.022614 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd7a1e-9a99-4a93-8a3a-8d723622280d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:55.025226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd7a1e-9a99-4a93-8a3a-8d723622280d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c2dd7a1e-9a99-4a93-8a3a-8d723622280d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:55.038317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:55.039376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 11:30:55.040199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:55.046116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 417/845] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:30:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:55.048655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd7a1e-9a99-4a93-8a3a-8d723622280d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:55.060646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a89bfb-ecd0-4c45-a5b2-ea3b171de391 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:55.103709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4373387-5165-4e17-80d6-de7b0b12d351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:55.104650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4373387-5165-4e17-80d6-de7b0b12d351 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.105304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4373387-5165-4e17-80d6-de7b0b12d351 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.105760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4373387-5165-4e17-80d6-de7b0b12d351 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.106579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4373387-5165-4e17-80d6-de7b0b12d351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:55.107069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4373387-5165-4e17-80d6-de7b0b12d351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:55.108624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4373387-5165-4e17-80d6-de7b0b12d351 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:55.125281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 429/846] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:55.126744 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4373387-5165-4e17-80d6-de7b0b12d351 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:55.162173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd7a1e-9a99-4a93-8a3a-8d723622280d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:55.202124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf695a23-bd85-47fa-825a-7ec5f05639a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:55.202840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf695a23-bd85-47fa-825a-7ec5f05639a2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.203247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf695a23-bd85-47fa-825a-7ec5f05639a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.203659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf695a23-bd85-47fa-825a-7ec5f05639a2 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.204558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf695a23-bd85-47fa-825a-7ec5f05639a2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:55.205005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf695a23-bd85-47fa-825a-7ec5f05639a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:55.207086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf695a23-bd85-47fa-825a-7ec5f05639a2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:55.217706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 418/847] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:30:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:55.218527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf695a23-bd85-47fa-825a-7ec5f05639a2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:55.238060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4373387-5165-4e17-80d6-de7b0b12d351 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:55.249764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:55.267215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:55.277367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7157fb7-a9a4-4d38-a92d-47e8150bb98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:55.278527 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7157fb7-a9a4-4d38-a92d-47e8150bb98e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.278990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7157fb7-a9a4-4d38-a92d-47e8150bb98e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.279555 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7157fb7-a9a4-4d38-a92d-47e8150bb98e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.288400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7157fb7-a9a4-4d38-a92d-47e8150bb98e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:55.291229 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7157fb7-a9a4-4d38-a92d-47e8150bb98e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b7157fb7-a9a4-4d38-a92d-47e8150bb98e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:55.307114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:55.308042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:55.309139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:55.309139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:55.309139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:55.309139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:55.309139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:55.309139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:55.309139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:55.309139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:55.315639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 430/848] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:55.317360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608afe8d-33b8-4939-b905-dfd4a8b72373 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.317790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608afe8d-33b8-4939-b905-dfd4a8b72373 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.318209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608afe8d-33b8-4939-b905-dfd4a8b72373 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.319616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 431/849] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:55.320530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608afe8d-33b8-4939-b905-dfd4a8b72373 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:55.328962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf695a23-bd85-47fa-825a-7ec5f05639a2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:55.363731 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbd2cc-8eb9-40a6-aed0-85583dff2097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:55.364335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbd2cc-8eb9-40a6-aed0-85583dff2097 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.364655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbd2cc-8eb9-40a6-aed0-85583dff2097 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.364937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbd2cc-8eb9-40a6-aed0-85583dff2097 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.365566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbd2cc-8eb9-40a6-aed0-85583dff2097 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:55.365874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbd2cc-8eb9-40a6-aed0-85583dff2097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:55.367241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbd2cc-8eb9-40a6-aed0-85583dff2097 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:55.372483 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 419/850] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:30:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:55.373479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1579d-42fd-4ad9-b08d-0f494389104a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.373765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1579d-42fd-4ad9-b08d-0f494389104a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.374040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1579d-42fd-4ad9-b08d-0f494389104a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.399640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cac1579d-42fd-4ad9-b08d-0f494389104a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:55.402452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cac1579d-42fd-4ad9-b08d-0f494389104a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:55.450803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608afe8d-33b8-4939-b905-dfd4a8b72373 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:55.494703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 420/851] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:30:55.496480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a9c5e-25cd-4369-9d28-6d8d7d4216d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:55.497140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a9c5e-25cd-4369-9d28-6d8d7d4216d0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.497433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a9c5e-25cd-4369-9d28-6d8d7d4216d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.497705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a9c5e-25cd-4369-9d28-6d8d7d4216d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.498393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a9c5e-25cd-4369-9d28-6d8d7d4216d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:55.498691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a9c5e-25cd-4369-9d28-6d8d7d4216d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:55.500105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a9c5e-25cd-4369-9d28-6d8d7d4216d0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:55.513586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 432/852] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:55.519211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1579d-42fd-4ad9-b08d-0f494389104a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:55.545285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:55.624285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1579d-42fd-4ad9-b08d-0f494389104a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:55.658886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23603d-8b39-45d8-afca-26bcba64eac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:55.659732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23603d-8b39-45d8-afca-26bcba64eac7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.660176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23603d-8b39-45d8-afca-26bcba64eac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.660629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23603d-8b39-45d8-afca-26bcba64eac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.661486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23603d-8b39-45d8-afca-26bcba64eac7 None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=3384c24dd081414bbf0a2e3d9f71126b) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:55.661932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23603d-8b39-45d8-afca-26bcba64eac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:55.663495 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23603d-8b39-45d8-afca-26bcba64eac7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:55.670365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 421/853] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:55 2022] GET /identity/v3/role_assignments?user.id=3384c24dd081414bbf0a2e3d9f71126b&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:55.791790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a9c5e-25cd-4369-9d28-6d8d7d4216d0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:55.845223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:55.901411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a9c5e-25cd-4369-9d28-6d8d7d4216d0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:55.909617 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12253-f09e-4f08-a286-ccb980df8772 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.910130 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12253-f09e-4f08-a286-ccb980df8772 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.910223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12253-f09e-4f08-a286-ccb980df8772 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.911746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 422/854] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:55.918033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4cca5-41b5-43cf-9813-687bf1763847 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.918470 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4cca5-41b5-43cf-9813-687bf1763847 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.918959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4cca5-41b5-43cf-9813-687bf1763847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.939331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fe783-edf8-4f2b-8d44-4cec75bad367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:55.940205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fe783-edf8-4f2b-8d44-4cec75bad367 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:55.940655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fe783-edf8-4f2b-8d44-4cec75bad367 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:55.941166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fe783-edf8-4f2b-8d44-4cec75bad367 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:55.942024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5eb4cca5-41b5-43cf-9813-687bf1763847 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:55.942341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6fe783-edf8-4f2b-8d44-4cec75bad367 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:55.943227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5eb4cca5-41b5-43cf-9813-687bf1763847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:55.944012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6fe783-edf8-4f2b-8d44-4cec75bad367 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-fb6fe783-edf8-4f2b-8d44-4cec75bad367 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:55.954024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:55.955200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 11:30:55.956298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:55.958523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 433/855] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Aug 31 11:30:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:55.965215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fe783-edf8-4f2b-8d44-4cec75bad367 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:56.018036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 423/856] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:30:56.084107 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fe783-edf8-4f2b-8d44-4cec75bad367 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:56.099828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:56.123632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ab731-a492-45ad-aa71-0284af6fdae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:56.124318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ab731-a492-45ad-aa71-0284af6fdae5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:56.124697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ab731-a492-45ad-aa71-0284af6fdae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:56.125033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ab731-a492-45ad-aa71-0284af6fdae5 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:56.125729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ab731-a492-45ad-aa71-0284af6fdae5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:56.126098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ab731-a492-45ad-aa71-0284af6fdae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:56.127527 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ab731-a492-45ad-aa71-0284af6fdae5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:56.132611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 434/857] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Aug 31 11:30:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:56.138705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4cca5-41b5-43cf-9813-687bf1763847 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:56.157702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:56.246332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4cca5-41b5-43cf-9813-687bf1763847 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:56.281664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5b126-1092-4b51-aa71-81dde35a8649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:56.282277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5b126-1092-4b51-aa71-81dde35a8649 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:56.282562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5b126-1092-4b51-aa71-81dde35a8649 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:56.282840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5b126-1092-4b51-aa71-81dde35a8649 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:56.283478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5b126-1092-4b51-aa71-81dde35a8649 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:56.283782 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5b126-1092-4b51-aa71-81dde35a8649 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:56.285191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5b126-1092-4b51-aa71-81dde35a8649 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:56.290203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 424/858] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Aug 31 11:30:56 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:56.315622 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ab731-a492-45ad-aa71-0284af6fdae5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:56.430813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ab731-a492-45ad-aa71-0284af6fdae5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:56.474245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319a9cc-ccea-40da-8e7a-9659f86bd790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:56.474863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319a9cc-ccea-40da-8e7a-9659f86bd790 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:56.475138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319a9cc-ccea-40da-8e7a-9659f86bd790 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:56.475400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319a9cc-ccea-40da-8e7a-9659f86bd790 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:56.486809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319a9cc-ccea-40da-8e7a-9659f86bd790 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8319a9cc-ccea-40da-8e7a-9659f86bd790 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:30:56.491691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:56.492708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:56.493980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 435/859] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:30:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:56.499233 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5b126-1092-4b51-aa71-81dde35a8649 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:56.583962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:56.607707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5b126-1092-4b51-aa71-81dde35a8649 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:56.643710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06ed90-e6f5-40b6-a32f-9430b00ce743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:56.644317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ed90-e6f5-40b6-a32f-9430b00ce743 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:56.644627 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ed90-e6f5-40b6-a32f-9430b00ce743 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:56.644901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ed90-e6f5-40b6-a32f-9430b00ce743 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:56.645569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06ed90-e6f5-40b6-a32f-9430b00ce743 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:56.645872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06ed90-e6f5-40b6-a32f-9430b00ce743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:56.647237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06ed90-e6f5-40b6-a32f-9430b00ce743 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:56.655665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 425/860] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:30:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:56.661221 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319a9cc-ccea-40da-8e7a-9659f86bd790 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:56.715425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:56.783434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319a9cc-ccea-40da-8e7a-9659f86bd790 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:56.821065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60326e-97dc-4638-9804-355b0150bfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:56.821729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60326e-97dc-4638-9804-355b0150bfea None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:56.822116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60326e-97dc-4638-9804-355b0150bfea None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:56.822557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60326e-97dc-4638-9804-355b0150bfea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:56.834723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60326e-97dc-4638-9804-355b0150bfea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:56.836194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60326e-97dc-4638-9804-355b0150bfea None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5c60326e-97dc-4638-9804-355b0150bfea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:56.852036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:56.853587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:56.855114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:56.855114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:56.855114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:56.855114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 436/861] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:56.859956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06ed90-e6f5-40b6-a32f-9430b00ce743 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:56.967070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06ed90-e6f5-40b6-a32f-9430b00ce743 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:57.002227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad8fc4-877c-4807-b3f2-f76689f09c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:57.002838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad8fc4-877c-4807-b3f2-f76689f09c10 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:57.003122 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad8fc4-877c-4807-b3f2-f76689f09c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:57.003386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad8fc4-877c-4807-b3f2-f76689f09c10 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:57.004034 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad8fc4-877c-4807-b3f2-f76689f09c10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:57.004339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad8fc4-877c-4807-b3f2-f76689f09c10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:57.005747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad8fc4-877c-4807-b3f2-f76689f09c10 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:57.015118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:57.018727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 426/862] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:57.024058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60326e-97dc-4638-9804-355b0150bfea None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:57.117179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:57.130783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60326e-97dc-4638-9804-355b0150bfea None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:57.166399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a042961-00cd-42c1-8de1-73e36d7fb819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:57.167081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a042961-00cd-42c1-8de1-73e36d7fb819 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:57.167404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a042961-00cd-42c1-8de1-73e36d7fb819 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:57.167677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a042961-00cd-42c1-8de1-73e36d7fb819 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:57.175415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a042961-00cd-42c1-8de1-73e36d7fb819 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:57.176875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a042961-00cd-42c1-8de1-73e36d7fb819 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9a042961-00cd-42c1-8de1-73e36d7fb819 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:30:57.188932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:57.191247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:57.193784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:57.193784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:57.193784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:30:57.193784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:30:57.193784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:30:57.193784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:30:57.193784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:30:57.193784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:30:57.193784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:30:57.193784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:57.193784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 437/863] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Aug 31 11:30:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:57.201512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad8fc4-877c-4807-b3f2-f76689f09c10 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:57.309271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad8fc4-877c-4807-b3f2-f76689f09c10 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:57.346540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e69d02-6702-4908-8be0-bb35d9f289c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:57.347520 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e69d02-6702-4908-8be0-bb35d9f289c0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:57.347994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e69d02-6702-4908-8be0-bb35d9f289c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:57.348445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e69d02-6702-4908-8be0-bb35d9f289c0 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:57.349387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e69d02-6702-4908-8be0-bb35d9f289c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:57.349889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e69d02-6702-4908-8be0-bb35d9f289c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:57.351507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e69d02-6702-4908-8be0-bb35d9f289c0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:57.370989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 427/864] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Aug 31 11:30:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:57.376957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a042961-00cd-42c1-8de1-73e36d7fb819 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:57.485897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a042961-00cd-42c1-8de1-73e36d7fb819 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:57.523570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662cf53-e59a-4f2e-9ad3-fcb4e65588bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:57.524542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662cf53-e59a-4f2e-9ad3-fcb4e65588bc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:57.525034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662cf53-e59a-4f2e-9ad3-fcb4e65588bc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:57.525485 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662cf53-e59a-4f2e-9ad3-fcb4e65588bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:57.545485 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:57.570973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6e08e-edf9-4717-9267-b1b94594f66d None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:57.571404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6e08e-edf9-4717-9267-b1b94594f66d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:57.572313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6e08e-edf9-4717-9267-b1b94594f66d None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:57.574251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 428/865] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:57.579094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0588924-b4c5-41f4-b4e3-53159845084d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:57.579484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0588924-b4c5-41f4-b4e3-53159845084d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:57.579863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0588924-b4c5-41f4-b4e3-53159845084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:57.598847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a0588924-b4c5-41f4-b4e3-53159845084d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:57.599348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662cf53-e59a-4f2e-9ad3-fcb4e65588bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:57.600276 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a0588924-b4c5-41f4-b4e3-53159845084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:57.601135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662cf53-e59a-4f2e-9ad3-fcb4e65588bc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:57.659349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:57.685552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 429/866] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:30:57.704368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7f9f0-93be-4d99-b6f0-82eab3b21480 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:57.704691 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7f9f0-93be-4d99-b6f0-82eab3b21480 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:57.705099 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7f9f0-93be-4d99-b6f0-82eab3b21480 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:57.706670 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 430/867] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:30:57.709606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 438/868] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:30:57 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:30:57.715790 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8cbc1-035c-4d07-90cc-7ddd9817847e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:57.716065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662cf53-e59a-4f2e-9ad3-fcb4e65588bc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:57.716170 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8cbc1-035c-4d07-90cc-7ddd9817847e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:57.716350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8cbc1-035c-4d07-90cc-7ddd9817847e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:57.737532 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-60b8cbc1-035c-4d07-90cc-7ddd9817847e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:57.739085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-60b8cbc1-035c-4d07-90cc-7ddd9817847e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:57.765663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:57.820641 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662cf53-e59a-4f2e-9ad3-fcb4e65588bc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:57.833005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 431/869] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:30:57.860681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846e576-e8c8-409a-8c80-baaa291b60fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:57.861360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846e576-e8c8-409a-8c80-baaa291b60fc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:57.861640 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846e576-e8c8-409a-8c80-baaa291b60fc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:57.861907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846e576-e8c8-409a-8c80-baaa291b60fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:57.873344 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846e576-e8c8-409a-8c80-baaa291b60fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:57.874914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846e576-e8c8-409a-8c80-baaa291b60fc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5846e576-e8c8-409a-8c80-baaa291b60fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:30:57.889926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:57.890883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:30:57.891770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:30:57.891770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:30:57.891770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:57.891770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 439/870] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:57.896938 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8cbc1-035c-4d07-90cc-7ddd9817847e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:57.971269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846e576-e8c8-409a-8c80-baaa291b60fc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:58.003098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8cbc1-035c-4d07-90cc-7ddd9817847e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:58.027180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:58.052255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfbdf0-3522-4d01-9c5c-ea0f1dbac2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:58.053220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfbdf0-3522-4d01-9c5c-ea0f1dbac2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:58.053727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfbdf0-3522-4d01-9c5c-ea0f1dbac2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:58.054209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfbdf0-3522-4d01-9c5c-ea0f1dbac2b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:58.055051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfbdf0-3522-4d01-9c5c-ea0f1dbac2b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:58.055548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfbdf0-3522-4d01-9c5c-ea0f1dbac2b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:58.058454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfbdf0-3522-4d01-9c5c-ea0f1dbac2b8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:58.075948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 432/871] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:58.084602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfbdf0-3522-4d01-9c5c-ea0f1dbac2b8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:58.084736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:58.088185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846e576-e8c8-409a-8c80-baaa291b60fc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:58.138701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958089d-d9ad-4d1b-be64-49aad7b1739f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:58.139921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958089d-d9ad-4d1b-be64-49aad7b1739f None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:58.140211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958089d-d9ad-4d1b-be64-49aad7b1739f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:58.140523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958089d-d9ad-4d1b-be64-49aad7b1739f None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:58.141175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958089d-d9ad-4d1b-be64-49aad7b1739f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:58.142535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958089d-d9ad-4d1b-be64-49aad7b1739f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:58.161925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 440/872] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:30:57 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:30:58.162830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958089d-d9ad-4d1b-be64-49aad7b1739f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:58.225051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfbdf0-3522-4d01-9c5c-ea0f1dbac2b8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:58.282882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958089d-d9ad-4d1b-be64-49aad7b1739f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:58.283555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b7ed8-e22b-43ca-86d2-26cd9eb950d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:58.284473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b7ed8-e22b-43ca-86d2-26cd9eb950d8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:58.285176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b7ed8-e22b-43ca-86d2-26cd9eb950d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:58.286550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b7ed8-e22b-43ca-86d2-26cd9eb950d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:58.297227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b7ed8-e22b-43ca-86d2-26cd9eb950d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:58.299892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b7ed8-e22b-43ca-86d2-26cd9eb950d8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-2d2b7ed8-e22b-43ca-86d2-26cd9eb950d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:30:58.318193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:58.319202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:58.320133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:30:58.320133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:30:58.320133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:30:58.320133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:30:58.320133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:30:58.320133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:30:58.320133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:30:58.320133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:30:58.322476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 433/873] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:30:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:30:58.325083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6dbf5-014d-4719-8906-94cfffcaed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:58.325804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6dbf5-014d-4719-8906-94cfffcaed9d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:58.325978 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717254-4a48-458b-8614-e1701e86bc0c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:58.326203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6dbf5-014d-4719-8906-94cfffcaed9d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:58.326770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6dbf5-014d-4719-8906-94cfffcaed9d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:58.327068 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717254-4a48-458b-8614-e1701e86bc0c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:58.327191 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6dbf5-014d-4719-8906-94cfffcaed9d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:58.327626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717254-4a48-458b-8614-e1701e86bc0c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:58.328606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6dbf5-014d-4719-8906-94cfffcaed9d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:58.329224 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 434/874] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:30:58.329739 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f717254-4a48-458b-8614-e1701e86bc0c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8ba6dbf5-014d-4719-8906-94cfffcaed9d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:30:58.342375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:30:58.343380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:30:58.344215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 441/875] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Aug 31 11:30:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:30:58.345129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55c9aa-c5a7-408d-9be4-772291b99f70 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:58.345129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55c9aa-c5a7-408d-9be4-772291b99f70 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:58.345249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55c9aa-c5a7-408d-9be4-772291b99f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:58.365009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8b55c9aa-c5a7-408d-9be4-772291b99f70 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:30:58.366216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8b55c9aa-c5a7-408d-9be4-772291b99f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:58.373753 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:58.384836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:58.446918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 442/876] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:30:58 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:30:58.447239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f717254-4a48-458b-8614-e1701e86bc0c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:58.448297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55c9aa-c5a7-408d-9be4-772291b99f70 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:58.501064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbc819-1387-4477-87a1-fc123e94a0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:58.502777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc819-1387-4477-87a1-fc123e94a0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:58.503413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc819-1387-4477-87a1-fc123e94a0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:58.504815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc819-1387-4477-87a1-fc123e94a0c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:58.506022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbc819-1387-4477-87a1-fc123e94a0c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:58.507237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbc819-1387-4477-87a1-fc123e94a0c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:58.509082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbc819-1387-4477-87a1-fc123e94a0c7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:58.526212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 435/877] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:30:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:58.531911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbc819-1387-4477-87a1-fc123e94a0c7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:58.564526 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55c9aa-c5a7-408d-9be4-772291b99f70 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:58.601859 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73805e-9a5f-42d5-a63e-7a4f56c650d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:58.602953 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73805e-9a5f-42d5-a63e-7a4f56c650d9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:58.604196 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73805e-9a5f-42d5-a63e-7a4f56c650d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:58.604731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73805e-9a5f-42d5-a63e-7a4f56c650d9 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:58.605571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73805e-9a5f-42d5-a63e-7a4f56c650d9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:58.606068 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73805e-9a5f-42d5-a63e-7a4f56c650d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:58.607731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73805e-9a5f-42d5-a63e-7a4f56c650d9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:58.613111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 443/878] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Aug 31 11:30:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:58.622038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73805e-9a5f-42d5-a63e-7a4f56c650d9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:58.638643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:58.642142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:58.655143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbc819-1387-4477-87a1-fc123e94a0c7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:58.705560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee06b2-7462-4042-a20c-a59693eeea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:58.707092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee06b2-7462-4042-a20c-a59693eeea22 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:58.707613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee06b2-7462-4042-a20c-a59693eeea22 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:58.708294 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee06b2-7462-4042-a20c-a59693eeea22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:58.709250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee06b2-7462-4042-a20c-a59693eeea22 None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=16216de2aa9c4a6388f94263444965df) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:58.709975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee06b2-7462-4042-a20c-a59693eeea22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:58.713152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee06b2-7462-4042-a20c-a59693eeea22 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:58.721728 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 436/879] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:30:58 2022] GET /identity/v3/role_assignments?user.id=16216de2aa9c4a6388f94263444965df&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:30:58.731450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee06b2-7462-4042-a20c-a59693eeea22 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:58.761225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73805e-9a5f-42d5-a63e-7a4f56c650d9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:58.813598 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523a84b-a68d-4f6d-84f3-dddc2e0cf7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:58.814539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523a84b-a68d-4f6d-84f3-dddc2e0cf7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:58.815033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523a84b-a68d-4f6d-84f3-dddc2e0cf7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:58.815489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523a84b-a68d-4f6d-84f3-dddc2e0cf7d6 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:58.816316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523a84b-a68d-4f6d-84f3-dddc2e0cf7d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:30:58.817658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523a84b-a68d-4f6d-84f3-dddc2e0cf7d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:58.819341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523a84b-a68d-4f6d-84f3-dddc2e0cf7d6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:58.827896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 444/880] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Aug 31 11:30:58 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:30:58.846050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee06b2-7462-4042-a20c-a59693eeea22 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:58.896075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae555d6-8896-495a-a593-6e6019326d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:58.896641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae555d6-8896-495a-a593-6e6019326d06 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:58.896924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae555d6-8896-495a-a593-6e6019326d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:58.897197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae555d6-8896-495a-a593-6e6019326d06 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:58.897866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae555d6-8896-495a-a593-6e6019326d06 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:58.899257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae555d6-8896-495a-a593-6e6019326d06 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:58.921038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 437/881] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:30:58 2022] POST /identity/v3/services => generated 269 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:31:00.153057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396370c3-5fb1-4f7e-9a67-5bad3195b2d9 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:00.153583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396370c3-5fb1-4f7e-9a67-5bad3195b2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:00.153583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396370c3-5fb1-4f7e-9a67-5bad3195b2d9 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:00.155186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 445/882] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:00.160691 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05fec9-b679-44cd-bb18-9a9cd587f451 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:00.161162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05fec9-b679-44cd-bb18-9a9cd587f451 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:00.161343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05fec9-b679-44cd-bb18-9a9cd587f451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:00.180100 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9b05fec9-b679-44cd-bb18-9a9cd587f451 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:00.181835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9b05fec9-b679-44cd-bb18-9a9cd587f451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:00.195942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:00.256868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 438/883] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:00.535290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396370c3-5fb1-4f7e-9a67-5bad3195b2d9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:00.622478 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:00.646154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396370c3-5fb1-4f7e-9a67-5bad3195b2d9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:00.682422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880ce39-d965-441e-87bc-e09287735d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:00.683250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ce39-d965-441e-87bc-e09287735d44 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:00.683537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ce39-d965-441e-87bc-e09287735d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:00.683810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ce39-d965-441e-87bc-e09287735d44 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:00.684413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880ce39-d965-441e-87bc-e09287735d44 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:00.685903 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880ce39-d965-441e-87bc-e09287735d44 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6880ce39-d965-441e-87bc-e09287735d44 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:00.695351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:00.696349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:00.697328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 446/884] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:31:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:00.704245 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05fec9-b679-44cd-bb18-9a9cd587f451 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:00.761187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed2ce2-2e08-4829-9ca9-a5958c981915 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:00.761487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed2ce2-2e08-4829-9ca9-a5958c981915 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:00.761832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed2ce2-2e08-4829-9ca9-a5958c981915 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:00.763384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 447/885] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:31:00.768293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a150a-c391-4cf3-b72e-92f6367fb496 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:00.768607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a150a-c391-4cf3-b72e-92f6367fb496 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:00.768881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a150a-c391-4cf3-b72e-92f6367fb496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:00.787692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e15a150a-c391-4cf3-b72e-92f6367fb496 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:00.788827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e15a150a-c391-4cf3-b72e-92f6367fb496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:00.810814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05fec9-b679-44cd-bb18-9a9cd587f451 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:00.849811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64104a72-d7e9-4c8d-be0d-fb3e2cbd8e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:00.850444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64104a72-d7e9-4c8d-be0d-fb3e2cbd8e40 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:00.850722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64104a72-d7e9-4c8d-be0d-fb3e2cbd8e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:00.850991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64104a72-d7e9-4c8d-be0d-fb3e2cbd8e40 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:00.851617 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64104a72-d7e9-4c8d-be0d-fb3e2cbd8e40 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:00.852020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64104a72-d7e9-4c8d-be0d-fb3e2cbd8e40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:00.853724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64104a72-d7e9-4c8d-be0d-fb3e2cbd8e40 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:00.859171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 439/886] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:31:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:00.860411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110851fd-6ec3-4aff-bd85-804e029c6b5c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:00.860724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110851fd-6ec3-4aff-bd85-804e029c6b5c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:00.860999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110851fd-6ec3-4aff-bd85-804e029c6b5c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:00.864579 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 440/887] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:00.864579 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110851fd-6ec3-4aff-bd85-804e029c6b5c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:00.875665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:00.877142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 448/888] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:00.878419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d46caa-8c18-4cb8-8cbb-d1734056ce7e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:00.878704 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d46caa-8c18-4cb8-8cbb-d1734056ce7e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:00.878972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d46caa-8c18-4cb8-8cbb-d1734056ce7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:00.900802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f8d46caa-8c18-4cb8-8cbb-d1734056ce7e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:00.902317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f8d46caa-8c18-4cb8-8cbb-d1734056ce7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:00.979706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110851fd-6ec3-4aff-bd85-804e029c6b5c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:00.980518 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 449/889] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:00.981964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad3c3b-949e-4771-9560-d4564414ce4b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:00.982250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad3c3b-949e-4771-9560-d4564414ce4b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:00.982522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad3c3b-949e-4771-9560-d4564414ce4b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:00.983888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 450/890] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:00.989222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85086a60-e369-4c26-b827-89b55ab76f5f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:00.989505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85086a60-e369-4c26-b827-89b55ab76f5f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:00.989782 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85086a60-e369-4c26-b827-89b55ab76f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:00.996666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:01.011042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-85086a60-e369-4c26-b827-89b55ab76f5f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:01.012482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-85086a60-e369-4c26-b827-89b55ab76f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:01.023986 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d3132-f857-45a1-996d-4b6bb0627f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:01.024626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3132-f857-45a1-996d-4b6bb0627f89 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:01.025028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3132-f857-45a1-996d-4b6bb0627f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:01.025416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3132-f857-45a1-996d-4b6bb0627f89 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:01.026079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d3132-f857-45a1-996d-4b6bb0627f89 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:01.026386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d3132-f857-45a1-996d-4b6bb0627f89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:01.027869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d3132-f857-45a1-996d-4b6bb0627f89 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:01.033404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 441/891] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:31:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:01.040188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d3132-f857-45a1-996d-4b6bb0627f89 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:01.099759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 451/892] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:01.125046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:01.149729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85086a60-e369-4c26-b827-89b55ab76f5f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:01.157908 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d3132-f857-45a1-996d-4b6bb0627f89 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:01.202844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2873a-8498-4f85-9e01-2c15ff19ae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:01.203478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2873a-8498-4f85-9e01-2c15ff19ae7d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:01.203712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2873a-8498-4f85-9e01-2c15ff19ae7d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:01.203985 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2873a-8498-4f85-9e01-2c15ff19ae7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:01.205236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2873a-8498-4f85-9e01-2c15ff19ae7d None admin] RBAC: Adding query filter params (interface=public, service_id=d1c3bd8b37564efbb071380211b23db6, region_id=RegionOne) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:01.205555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2873a-8498-4f85-9e01-2c15ff19ae7d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:01.207174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2873a-8498-4f85-9e01-2c15ff19ae7d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:01.213766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 442/893] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Aug 31 11:31:01 2022] GET /identity/v3/endpoints?service_id=d1c3bd8b37564efbb071380211b23db6&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:01.219215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2873a-8498-4f85-9e01-2c15ff19ae7d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:01.267750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85086a60-e369-4c26-b827-89b55ab76f5f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:01.300437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:01.307267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3709c62-84a1-48d4-bd73-a31d95a5949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:01.307867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3709c62-84a1-48d4-bd73-a31d95a5949e None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:01.308144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3709c62-84a1-48d4-bd73-a31d95a5949e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:01.308411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3709c62-84a1-48d4-bd73-a31d95a5949e None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:01.309093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3709c62-84a1-48d4-bd73-a31d95a5949e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:01.310463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3709c62-84a1-48d4-bd73-a31d95a5949e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:01.325443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 452/894] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:31:01 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:31:01.326522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3709c62-84a1-48d4-bd73-a31d95a5949e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:01.330757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2873a-8498-4f85-9e01-2c15ff19ae7d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:01.373052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:01.378291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74368895-7021-4765-8950-ac0c5eaedb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:01.379906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74368895-7021-4765-8950-ac0c5eaedb65 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:01.380197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74368895-7021-4765-8950-ac0c5eaedb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:01.380476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74368895-7021-4765-8950-ac0c5eaedb65 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:01.381211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74368895-7021-4765-8950-ac0c5eaedb65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:01.382684 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74368895-7021-4765-8950-ac0c5eaedb65 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:01.391297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 443/895] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Aug 31 11:31:01 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:31:01.392178 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74368895-7021-4765-8950-ac0c5eaedb65 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:01.457579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3709c62-84a1-48d4-bd73-a31d95a5949e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:01.498518 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1838e-0b85-415e-9858-6f124c04b66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:01.499241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1838e-0b85-415e-9858-6f124c04b66a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:01.499527 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1838e-0b85-415e-9858-6f124c04b66a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:01.499793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1838e-0b85-415e-9858-6f124c04b66a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:01.513353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1838e-0b85-415e-9858-6f124c04b66a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cac1838e-0b85-415e-9858-6f124c04b66a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:01.518774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:01.519813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:01.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 453/896] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Aug 31 11:31:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:01.526656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1838e-0b85-415e-9858-6f124c04b66a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:01.533425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74368895-7021-4765-8950-ac0c5eaedb65 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:01.579141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9e75f-f5b9-453c-a7f4-abcd16b8846e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:01.579920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9e75f-f5b9-453c-a7f4-abcd16b8846e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:01.580201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9e75f-f5b9-453c-a7f4-abcd16b8846e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:01.580463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9e75f-f5b9-453c-a7f4-abcd16b8846e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:01.581388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9e75f-f5b9-453c-a7f4-abcd16b8846e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:01.582568 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9e75f-f5b9-453c-a7f4-abcd16b8846e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d5b9e75f-f5b9-453c-a7f4-abcd16b8846e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:01.596915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:01.598155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 11:31:01.599271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:01.600918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:01.604638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 444/897] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Aug 31 11:31:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:01.607253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9e75f-f5b9-453c-a7f4-abcd16b8846e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:01.656954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1838e-0b85-415e-9858-6f124c04b66a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:01.699987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:01.709395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2638f21-7c41-49c9-84c1-e9a132d9abc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:01.710746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2638f21-7c41-49c9-84c1-e9a132d9abc6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:01.711132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2638f21-7c41-49c9-84c1-e9a132d9abc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:01.711507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2638f21-7c41-49c9-84c1-e9a132d9abc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:01.713136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2638f21-7c41-49c9-84c1-e9a132d9abc6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:01.713539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2638f21-7c41-49c9-84c1-e9a132d9abc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:01.716178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2638f21-7c41-49c9-84c1-e9a132d9abc6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:01.731507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 454/898] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Aug 31 11:31:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:01.733119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2638f21-7c41-49c9-84c1-e9a132d9abc6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:01.753878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9e75f-f5b9-453c-a7f4-abcd16b8846e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:01.799548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8d11d-25f1-462c-9694-ceca35ee43cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:01.800218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8d11d-25f1-462c-9694-ceca35ee43cf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:01.800531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8d11d-25f1-462c-9694-ceca35ee43cf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:01.800807 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8d11d-25f1-462c-9694-ceca35ee43cf None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:01.801700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8d11d-25f1-462c-9694-ceca35ee43cf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:01.802012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8d11d-25f1-462c-9694-ceca35ee43cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:01.804055 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8d11d-25f1-462c-9694-ceca35ee43cf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:01.812571 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 445/899] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Aug 31 11:31:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:01.817257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8d11d-25f1-462c-9694-ceca35ee43cf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:01.858831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2638f21-7c41-49c9-84c1-e9a132d9abc6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:01.873823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:01.896409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726829b1-d651-45e9-b91f-657e40d54e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:01.897160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726829b1-d651-45e9-b91f-657e40d54e66 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:01.897508 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726829b1-d651-45e9-b91f-657e40d54e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:01.897843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726829b1-d651-45e9-b91f-657e40d54e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:01.910029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726829b1-d651-45e9-b91f-657e40d54e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:01.911710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726829b1-d651-45e9-b91f-657e40d54e66 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-726829b1-d651-45e9-b91f-657e40d54e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:01.926697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:01.927697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:01.929046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:01.929046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:01.929046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:01.929046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 455/900] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:01.933784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726829b1-d651-45e9-b91f-657e40d54e66 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:01.964111 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8d11d-25f1-462c-9694-ceca35ee43cf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:02.000008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:02.015487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6204256-0b17-4474-be3d-953538497032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:02.016715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6204256-0b17-4474-be3d-953538497032 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:02.017204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6204256-0b17-4474-be3d-953538497032 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:02.017708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6204256-0b17-4474-be3d-953538497032 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:02.018815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6204256-0b17-4474-be3d-953538497032 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:02.019350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6204256-0b17-4474-be3d-953538497032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:02.021847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6204256-0b17-4474-be3d-953538497032 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:02.034250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 446/901] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Aug 31 11:31:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:02.035831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6204256-0b17-4474-be3d-953538497032 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:02.062653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726829b1-d651-45e9-b91f-657e40d54e66 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:02.107989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f9840-83ec-4ae5-957d-c3898c968e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:02.108658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9840-83ec-4ae5-957d-c3898c968e85 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:02.108993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9840-83ec-4ae5-957d-c3898c968e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:02.109214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9840-83ec-4ae5-957d-c3898c968e85 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:02.109918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f9840-83ec-4ae5-957d-c3898c968e85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:02.110224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f9840-83ec-4ae5-957d-c3898c968e85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:02.111714 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f9840-83ec-4ae5-957d-c3898c968e85 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:02.135406 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 456/902] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:02.137905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f9840-83ec-4ae5-957d-c3898c968e85 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:02.171898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:02.180724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6204256-0b17-4474-be3d-953538497032 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:02.225079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1c402-3d8d-4163-bc61-9bbd5587c413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:02.226848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1c402-3d8d-4163-bc61-9bbd5587c413 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:02.227289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1c402-3d8d-4163-bc61-9bbd5587c413 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:02.227711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1c402-3d8d-4163-bc61-9bbd5587c413 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:02.229750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1c402-3d8d-4163-bc61-9bbd5587c413 None admin] RBAC: Adding query filter params (interface=public, service_id=7e9d979011e94076b13e73a72f4d50b9, region_id=RegionOne) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:02.230642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1c402-3d8d-4163-bc61-9bbd5587c413 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:02.232440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1c402-3d8d-4163-bc61-9bbd5587c413 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:02.244292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 447/903] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Aug 31 11:31:02 2022] GET /identity/v3/endpoints?service_id=7e9d979011e94076b13e73a72f4d50b9&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:02.246296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1c402-3d8d-4163-bc61-9bbd5587c413 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:02.257707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f9840-83ec-4ae5-957d-c3898c968e85 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:02.297612 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094b777-9871-486c-856e-52e7c6d01c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:02.298289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094b777-9871-486c-856e-52e7c6d01c3f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:02.298566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094b777-9871-486c-856e-52e7c6d01c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:02.298846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094b777-9871-486c-856e-52e7c6d01c3f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:02.309365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094b777-9871-486c-856e-52e7c6d01c3f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:02.310991 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094b777-9871-486c-856e-52e7c6d01c3f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:02.317341 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0094b777-9871-486c-856e-52e7c6d01c3f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:31:02.327895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:02.329160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:02.330245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:02.330245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:02.330245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:31:02.330245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:31:02.330245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:31:02.330245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:31:02.330245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:31:02.330245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:31:02.330245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:31:02.330245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:02.338112 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 457/904] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Aug 31 11:31:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:02.339336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094b777-9871-486c-856e-52e7c6d01c3f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:02.373498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1c402-3d8d-4163-bc61-9bbd5587c413 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:02.410850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a115890-667c-413d-965a-9b5a1598d609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:02.411458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a115890-667c-413d-965a-9b5a1598d609 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:02.411784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a115890-667c-413d-965a-9b5a1598d609 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:02.412064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a115890-667c-413d-965a-9b5a1598d609 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:02.412703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a115890-667c-413d-965a-9b5a1598d609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:02.414070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a115890-667c-413d-965a-9b5a1598d609 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:02.421078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 448/905] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Aug 31 11:31:02 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:31:02.464031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:02.477360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094b777-9871-486c-856e-52e7c6d01c3f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:02.519310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9902535-e881-4893-8771-78e31f96d048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:02.519943 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9902535-e881-4893-8771-78e31f96d048 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:02.520222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9902535-e881-4893-8771-78e31f96d048 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:02.520595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9902535-e881-4893-8771-78e31f96d048 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:02.521410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9902535-e881-4893-8771-78e31f96d048 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:02.521722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9902535-e881-4893-8771-78e31f96d048 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:02.523170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9902535-e881-4893-8771-78e31f96d048 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:02.543984 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 458/906] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Aug 31 11:31:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:02.549778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a115890-667c-413d-965a-9b5a1598d609 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:02.660825 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a115890-667c-413d-965a-9b5a1598d609 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:02.685797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:02.706460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed4117-68e3-4374-bcfe-3a5aaa37064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:02.707079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed4117-68e3-4374-bcfe-3a5aaa37064a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:02.707361 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed4117-68e3-4374-bcfe-3a5aaa37064a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:02.707637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed4117-68e3-4374-bcfe-3a5aaa37064a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:02.781180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed4117-68e3-4374-bcfe-3a5aaa37064a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:02.782641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed4117-68e3-4374-bcfe-3a5aaa37064a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:02.893245 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 449/907] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:02 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:31:02.899277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9902535-e881-4893-8771-78e31f96d048 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:03.010588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9902535-e881-4893-8771-78e31f96d048 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:03.049002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc9cfb-0939-4d96-9461-40a780b87e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:03.050088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc9cfb-0939-4d96-9461-40a780b87e07 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:03.050594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc9cfb-0939-4d96-9461-40a780b87e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:03.051080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc9cfb-0939-4d96-9461-40a780b87e07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:03.062811 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc9cfb-0939-4d96-9461-40a780b87e07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:03.064542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc9cfb-0939-4d96-9461-40a780b87e07 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:03.067741 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b3dc9cfb-0939-4d96-9461-40a780b87e07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:03.082348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:03.083339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:03.084832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:03.084832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:03.084832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:03.084832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 459/908] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:03.090154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed4117-68e3-4374-bcfe-3a5aaa37064a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:03.108946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:03.200770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed4117-68e3-4374-bcfe-3a5aaa37064a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:03.238049 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a5ae9-28c0-4339-b157-7a71006b6df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:03.238713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a5ae9-28c0-4339-b157-7a71006b6df1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:03.238950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a5ae9-28c0-4339-b157-7a71006b6df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:03.239209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a5ae9-28c0-4339-b157-7a71006b6df1 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:03.239867 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a5ae9-28c0-4339-b157-7a71006b6df1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:03.240234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a5ae9-28c0-4339-b157-7a71006b6df1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:03.241655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a5ae9-28c0-4339-b157-7a71006b6df1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:03.254413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 450/909] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:03.260850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc9cfb-0939-4d96-9461-40a780b87e07 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:03.357973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4cb7d-9ccd-4ec4-8849-846f3d115578 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:03.358338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4cb7d-9ccd-4ec4-8849-846f3d115578 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:03.358531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4cb7d-9ccd-4ec4-8849-846f3d115578 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:03.360219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 451/910] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:31:03.365015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdb56d-4cbf-4388-b813-16c6acb309c7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:03.365279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdb56d-4cbf-4388-b813-16c6acb309c7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:03.365558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdb56d-4cbf-4388-b813-16c6acb309c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:03.374024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc9cfb-0939-4d96-9461-40a780b87e07 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:03.383183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-47fdb56d-4cbf-4388-b813-16c6acb309c7 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:03.384909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-47fdb56d-4cbf-4388-b813-16c6acb309c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:03.410716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7fafa-4717-4852-8c23-83ed66b24360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:03.411332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7fafa-4717-4852-8c23-83ed66b24360 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:03.411609 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7fafa-4717-4852-8c23-83ed66b24360 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:03.412129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7fafa-4717-4852-8c23-83ed66b24360 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:03.418627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7fafa-4717-4852-8c23-83ed66b24360 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:03.420047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7fafa-4717-4852-8c23-83ed66b24360 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-75f7fafa-4717-4852-8c23-83ed66b24360 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:31:03.431452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:03.432455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:03.433390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:31:03.433390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:31:03.433390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:31:03.433390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:31:03.433390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:31:03.433390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:31:03.433390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:03.433390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:03.433390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 460/911] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:31:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:03.434474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc73933-f98d-4307-a8bd-c37afcee0432 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:03.434474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc73933-f98d-4307-a8bd-c37afcee0432 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:03.434645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc73933-f98d-4307-a8bd-c37afcee0432 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:03.435928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 461/912] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:03.440438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc73933-f98d-4307-a8bd-c37afcee0432 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:03.461417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 452/913] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:03.465806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525bac4-b8d8-4a4e-a560-920c433f29e2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:03.467267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525bac4-b8d8-4a4e-a560-920c433f29e2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:03.467911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525bac4-b8d8-4a4e-a560-920c433f29e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:03.493696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3525bac4-b8d8-4a4e-a560-920c433f29e2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:03.495501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3525bac4-b8d8-4a4e-a560-920c433f29e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:03.499554 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:03.513364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:03.571492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc73933-f98d-4307-a8bd-c37afcee0432 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:03.588514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 453/914] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:03.612046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007d572-7202-442b-a0a0-57b70038203b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:03.612753 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007d572-7202-442b-a0a0-57b70038203b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:03.613027 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007d572-7202-442b-a0a0-57b70038203b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:03.613295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007d572-7202-442b-a0a0-57b70038203b None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:03.613983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007d572-7202-442b-a0a0-57b70038203b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:03.614284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007d572-7202-442b-a0a0-57b70038203b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:03.615743 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007d572-7202-442b-a0a0-57b70038203b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:03.628663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 462/915] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:03.634523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525bac4-b8d8-4a4e-a560-920c433f29e2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:03.742185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007d572-7202-442b-a0a0-57b70038203b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:03.753288 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525bac4-b8d8-4a4e-a560-920c433f29e2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:03.798172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee1ee-6784-423b-bcb9-0e5fcb5195c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:03.798700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee1ee-6784-423b-bcb9-0e5fcb5195c0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:03.799261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee1ee-6784-423b-bcb9-0e5fcb5195c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:03.799540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee1ee-6784-423b-bcb9-0e5fcb5195c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:03.800298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee1ee-6784-423b-bcb9-0e5fcb5195c0 None admin] RBAC: Adding query filter params (role.id=455a7f6a98b44607998cf25d8dec97cc, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=16216de2aa9c4a6388f94263444965df) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:03.801086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee1ee-6784-423b-bcb9-0e5fcb5195c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:03.802820 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee1ee-6784-423b-bcb9-0e5fcb5195c0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:03.809353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:03.813136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 454/916] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:31:03 2022] GET /identity/v3/role_assignments?user.id=16216de2aa9c4a6388f94263444965df&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=455a7f6a98b44607998cf25d8dec97cc => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:03.870677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007d572-7202-442b-a0a0-57b70038203b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:03.886212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee1ee-6784-423b-bcb9-0e5fcb5195c0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:03.889740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:03.910869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438945d-8cfb-4de7-9220-d049b47dbdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:03.911511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438945d-8cfb-4de7-9220-d049b47dbdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:03.911787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438945d-8cfb-4de7-9220-d049b47dbdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:03.912057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438945d-8cfb-4de7-9220-d049b47dbdd1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:03.912676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438945d-8cfb-4de7-9220-d049b47dbdd1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:03.914053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438945d-8cfb-4de7-9220-d049b47dbdd1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8438945d-8cfb-4de7-9220-d049b47dbdd1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:03.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:03.925448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:03.926331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 463/917] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:31:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:03.932316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438945d-8cfb-4de7-9220-d049b47dbdd1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:04.014127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee1ee-6784-423b-bcb9-0e5fcb5195c0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:04.054116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf04b32-9b6f-46df-96b5-f943b74c1b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:04.054664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf04b32-9b6f-46df-96b5-f943b74c1b73 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:04.054944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf04b32-9b6f-46df-96b5-f943b74c1b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:04.055215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf04b32-9b6f-46df-96b5-f943b74c1b73 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:04.055800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf04b32-9b6f-46df-96b5-f943b74c1b73 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:04.057517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf04b32-9b6f-46df-96b5-f943b74c1b73 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:04.059107 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438945d-8cfb-4de7-9220-d049b47dbdd1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-fdf04b32-9b6f-46df-96b5-f943b74c1b73 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:04.069298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:04.070343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:04.071172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:04.075441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 455/918] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Aug 31 11:31:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:04.077087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf04b32-9b6f-46df-96b5-f943b74c1b73 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:04.111572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7db50-5fbd-4161-ae10-b80de118e9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:04.112586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7db50-5fbd-4161-ae10-b80de118e9ab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:04.113273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7db50-5fbd-4161-ae10-b80de118e9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:04.114652 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7db50-5fbd-4161-ae10-b80de118e9ab None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:04.115488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7db50-5fbd-4161-ae10-b80de118e9ab None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:04.116008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7db50-5fbd-4161-ae10-b80de118e9ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:04.117607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7db50-5fbd-4161-ae10-b80de118e9ab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:04.125313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 464/919] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:31:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:04.130710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7db50-5fbd-4161-ae10-b80de118e9ab None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:04.147023 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:04.186579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:04.189367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf04b32-9b6f-46df-96b5-f943b74c1b73 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:04.226067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6948e3-3dae-47cb-9cde-921ff6ee8d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:04.227042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6948e3-3dae-47cb-9cde-921ff6ee8d97 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:04.227542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6948e3-3dae-47cb-9cde-921ff6ee8d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:04.228070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6948e3-3dae-47cb-9cde-921ff6ee8d97 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:04.229096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6948e3-3dae-47cb-9cde-921ff6ee8d97 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:04.229602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6948e3-3dae-47cb-9cde-921ff6ee8d97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:04.231256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6948e3-3dae-47cb-9cde-921ff6ee8d97 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:04.241267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 456/920] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Aug 31 11:31:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:04.242851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6948e3-3dae-47cb-9cde-921ff6ee8d97 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:04.248784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7db50-5fbd-4161-ae10-b80de118e9ab None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:04.295449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f76e3-c272-44aa-a173-f7dbabccfbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:04.296851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f76e3-c272-44aa-a173-f7dbabccfbac None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:04.297623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f76e3-c272-44aa-a173-f7dbabccfbac None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:04.298104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f76e3-c272-44aa-a173-f7dbabccfbac None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:04.299178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f76e3-c272-44aa-a173-f7dbabccfbac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:04.300158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f76e3-c272-44aa-a173-f7dbabccfbac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:04.302115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f76e3-c272-44aa-a173-f7dbabccfbac None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:04.309201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 465/921] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Aug 31 11:31:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:04.315512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f76e3-c272-44aa-a173-f7dbabccfbac None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:04.364844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6948e3-3dae-47cb-9cde-921ff6ee8d97 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:04.394467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:04.403958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2952f76-d123-4f16-b096-080f8926e173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:04.404916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2952f76-d123-4f16-b096-080f8926e173 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:04.405496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2952f76-d123-4f16-b096-080f8926e173 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:04.405997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2952f76-d123-4f16-b096-080f8926e173 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:04.406837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2952f76-d123-4f16-b096-080f8926e173 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:04.407434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2952f76-d123-4f16-b096-080f8926e173 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:04.409183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2952f76-d123-4f16-b096-080f8926e173 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:04.415013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 457/922] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Aug 31 11:31:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:04.420643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2952f76-d123-4f16-b096-080f8926e173 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:04.436991 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f76e3-c272-44aa-a173-f7dbabccfbac None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:04.447180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:04.474142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88297a97-1340-49df-ab18-f4b9d76c8a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:04.474740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297a97-1340-49df-ab18-f4b9d76c8a30 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:04.475016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297a97-1340-49df-ab18-f4b9d76c8a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:04.475286 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297a97-1340-49df-ab18-f4b9d76c8a30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:04.476109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88297a97-1340-49df-ab18-f4b9d76c8a30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:04.477581 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88297a97-1340-49df-ab18-f4b9d76c8a30 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:04.508520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 466/923] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:31:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:04.544238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2952f76-d123-4f16-b096-080f8926e173 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:04.576451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da80cdf-a1d0-40ca-994e-4d11a84fc397 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:04.576831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da80cdf-a1d0-40ca-994e-4d11a84fc397 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:04.577046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da80cdf-a1d0-40ca-994e-4d11a84fc397 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:04.578532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 467/924] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:31:04.583815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963cecf-696f-45df-a84f-0bc7fb450271 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:04.584169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963cecf-696f-45df-a84f-0bc7fb450271 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:04.584374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963cecf-696f-45df-a84f-0bc7fb450271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:04.597855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b89db-0f06-45ae-b3de-4c222b8bdc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:04.599092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b89db-0f06-45ae-b3de-4c222b8bdc85 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:04.599375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b89db-0f06-45ae-b3de-4c222b8bdc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:04.599651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b89db-0f06-45ae-b3de-4c222b8bdc85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:04.600784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b89db-0f06-45ae-b3de-4c222b8bdc85 None admin] RBAC: Adding query filter params (interface=public, service_id=08fcfb8bc61c46309434b7f83c40be79, region_id=RegionOne) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:04.601103 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b89db-0f06-45ae-b3de-4c222b8bdc85 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:04.602572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b89db-0f06-45ae-b3de-4c222b8bdc85 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:04.604681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7963cecf-696f-45df-a84f-0bc7fb450271 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:04.605926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7963cecf-696f-45df-a84f-0bc7fb450271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:04.610805 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 458/925] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Aug 31 11:31:04 2022] GET /identity/v3/endpoints?service_id=08fcfb8bc61c46309434b7f83c40be79&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:04.617333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b89db-0f06-45ae-b3de-4c222b8bdc85 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:04.694092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:04.694855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 468/926] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:04.746288 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b89db-0f06-45ae-b3de-4c222b8bdc85 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:04.781438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2f189-61fb-4f8e-bdad-08cee303ce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:04.782047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2f189-61fb-4f8e-bdad-08cee303ce63 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:04.782321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2f189-61fb-4f8e-bdad-08cee303ce63 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:04.782595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2f189-61fb-4f8e-bdad-08cee303ce63 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:04.783199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2f189-61fb-4f8e-bdad-08cee303ce63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:04.784599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2f189-61fb-4f8e-bdad-08cee303ce63 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:04.791891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 459/927] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Aug 31 11:31:04 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:31:04.971306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963cecf-696f-45df-a84f-0bc7fb450271 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:05.068930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:05.077129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963cecf-696f-45df-a84f-0bc7fb450271 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:05.114023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b21ba-c9a3-4d1e-8a46-75a6d17ac5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:05.114666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b21ba-c9a3-4d1e-8a46-75a6d17ac5cd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:05.114945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b21ba-c9a3-4d1e-8a46-75a6d17ac5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:05.115218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b21ba-c9a3-4d1e-8a46-75a6d17ac5cd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:05.115807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b21ba-c9a3-4d1e-8a46-75a6d17ac5cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:05.117196 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b21ba-c9a3-4d1e-8a46-75a6d17ac5cd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-910b21ba-c9a3-4d1e-8a46-75a6d17ac5cd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:05.126755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:05.127705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:05.129009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 469/928] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Aug 31 11:31:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:05.134145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2f189-61fb-4f8e-bdad-08cee303ce63 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:05.239748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2f189-61fb-4f8e-bdad-08cee303ce63 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:05.258162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:05.278844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16896b-d376-41ce-8bc1-0720b7843dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:05.279829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16896b-d376-41ce-8bc1-0720b7843dbb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:05.280326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16896b-d376-41ce-8bc1-0720b7843dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:05.280829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16896b-d376-41ce-8bc1-0720b7843dbb None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:05.281686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16896b-d376-41ce-8bc1-0720b7843dbb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:05.282212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16896b-d376-41ce-8bc1-0720b7843dbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:05.283825 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16896b-d376-41ce-8bc1-0720b7843dbb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:05.289536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 460/929] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Aug 31 11:31:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:05.300620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b21ba-c9a3-4d1e-8a46-75a6d17ac5cd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:05.403670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b21ba-c9a3-4d1e-8a46-75a6d17ac5cd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:05.438234 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf045262-346e-4fd3-a8a7-63ef4761b3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:05.438892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf045262-346e-4fd3-a8a7-63ef4761b3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:05.439120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf045262-346e-4fd3-a8a7-63ef4761b3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:05.439365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf045262-346e-4fd3-a8a7-63ef4761b3b2 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:05.439987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf045262-346e-4fd3-a8a7-63ef4761b3b2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:05.440284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf045262-346e-4fd3-a8a7-63ef4761b3b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:05.441923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf045262-346e-4fd3-a8a7-63ef4761b3b2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:05.447450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 470/930] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Aug 31 11:31:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:05.453057 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16896b-d376-41ce-8bc1-0720b7843dbb None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:05.560596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16896b-d376-41ce-8bc1-0720b7843dbb None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:05.596223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701454-3a25-490f-8849-f0bca0b65dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:05.596851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701454-3a25-490f-8849-f0bca0b65dbb None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:05.597120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701454-3a25-490f-8849-f0bca0b65dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:05.597392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701454-3a25-490f-8849-f0bca0b65dbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:05.598061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701454-3a25-490f-8849-f0bca0b65dbb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:05.599421 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701454-3a25-490f-8849-f0bca0b65dbb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:05.622996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 461/931] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:31:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:05.778407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096a7db-b9b1-4ad0-a137-2ab8a6ebe86f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:05.778621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096a7db-b9b1-4ad0-a137-2ab8a6ebe86f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:05.778898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096a7db-b9b1-4ad0-a137-2ab8a6ebe86f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:05.780781 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 471/932] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:05.786717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2d93a-f170-4e97-8747-6f84cb83eada None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:05.787008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2d93a-f170-4e97-8747-6f84cb83eada None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:05.787278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2d93a-f170-4e97-8747-6f84cb83eada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:05.807849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-daa2d93a-f170-4e97-8747-6f84cb83eada None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:05.809188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-daa2d93a-f170-4e97-8747-6f84cb83eada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:05.843786 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:05.893489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 462/933] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:06.169760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096a7db-b9b1-4ad0-a137-2ab8a6ebe86f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:06.197163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:06.281496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096a7db-b9b1-4ad0-a137-2ab8a6ebe86f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:06.317153 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa025c-0ee5-4ea1-a12c-d8072362a9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:06.318147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa025c-0ee5-4ea1-a12c-d8072362a9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:06.318623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa025c-0ee5-4ea1-a12c-d8072362a9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:06.319087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa025c-0ee5-4ea1-a12c-d8072362a9d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:06.330686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa025c-0ee5-4ea1-a12c-d8072362a9d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c5aa025c-0ee5-4ea1-a12c-d8072362a9d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:06.335930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:06.336922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:06.338350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:06.341111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 472/934] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Aug 31 11:31:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:06.346872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2d93a-f170-4e97-8747-6f84cb83eada None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:06.454364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2d93a-f170-4e97-8747-6f84cb83eada None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:06.488910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaed3a3-a715-4ba2-9126-b60121234f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:06.489526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaed3a3-a715-4ba2-9126-b60121234f67 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:06.489799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaed3a3-a715-4ba2-9126-b60121234f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:06.490069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaed3a3-a715-4ba2-9126-b60121234f67 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:06.490734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaed3a3-a715-4ba2-9126-b60121234f67 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:06.491033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaed3a3-a715-4ba2-9126-b60121234f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:06.492421 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaed3a3-a715-4ba2-9126-b60121234f67 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:06.501377 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 463/935] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Aug 31 11:31:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:06.510338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa025c-0ee5-4ea1-a12c-d8072362a9d5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:06.600909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:06.622184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa025c-0ee5-4ea1-a12c-d8072362a9d5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:06.656678 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d0015-cad0-44dd-8be0-dfaacc55670c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:06.656990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d0015-cad0-44dd-8be0-dfaacc55670c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:06.657221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d0015-cad0-44dd-8be0-dfaacc55670c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:06.659416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 464/936] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:31:06.663275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafac861-5135-4f43-a5d7-05b61f46e2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:06.664224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafac861-5135-4f43-a5d7-05b61f46e2ec None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:06.664791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafac861-5135-4f43-a5d7-05b61f46e2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:06.665258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafac861-5135-4f43-a5d7-05b61f46e2ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:06.666209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571720aa-8554-4f5a-b046-c84ec4b35761 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:06.666536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571720aa-8554-4f5a-b046-c84ec4b35761 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:06.666770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571720aa-8554-4f5a-b046-c84ec4b35761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:06.687227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-571720aa-8554-4f5a-b046-c84ec4b35761 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:06.688371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-571720aa-8554-4f5a-b046-c84ec4b35761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:06.689714 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafac861-5135-4f43-a5d7-05b61f46e2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:06.692677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafac861-5135-4f43-a5d7-05b61f46e2ec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cafac861-5135-4f43-a5d7-05b61f46e2ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:06.711873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:06.713404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:06.714400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:06.714400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:06.714400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:06.715670 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:06.715758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 473/937] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:06.719763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafac861-5135-4f43-a5d7-05b61f46e2ec None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:06.773252 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 465/938] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:06.775534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cfac0-89df-4a77-ad14-88b722655063 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:06.776024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cfac0-89df-4a77-ad14-88b722655063 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:06.776517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cfac0-89df-4a77-ad14-88b722655063 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:06.778332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 466/939] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:06.788641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d8d92-3890-427e-b24c-91c98ce18c45 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:06.788641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d8d92-3890-427e-b24c-91c98ce18c45 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:06.788641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d8d92-3890-427e-b24c-91c98ce18c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:06.805030 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f63d8d92-3890-427e-b24c-91c98ce18c45 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:06.806704 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f63d8d92-3890-427e-b24c-91c98ce18c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:06.831663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafac861-5135-4f43-a5d7-05b61f46e2ec None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:06.868239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3c29f-e254-466d-9454-2b0856a27288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:06.869171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3c29f-e254-466d-9454-2b0856a27288 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:06.869610 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3c29f-e254-466d-9454-2b0856a27288 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:06.870040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3c29f-e254-466d-9454-2b0856a27288 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:06.870828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3c29f-e254-466d-9454-2b0856a27288 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:06.871280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3c29f-e254-466d-9454-2b0856a27288 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:06.872804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3c29f-e254-466d-9454-2b0856a27288 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:06.874651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:06.887421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 474/940] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:06.896533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3c29f-e254-466d-9454-2b0856a27288 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:06.900711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 467/941] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:07.000231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3c29f-e254-466d-9454-2b0856a27288 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:07.035459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cca1d5-2ec1-4ace-9dad-a34e029d2657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:07.036360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cca1d5-2ec1-4ace-9dad-a34e029d2657 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.036854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cca1d5-2ec1-4ace-9dad-a34e029d2657 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.037320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cca1d5-2ec1-4ace-9dad-a34e029d2657 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.045269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cca1d5-2ec1-4ace-9dad-a34e029d2657 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:07.046898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cca1d5-2ec1-4ace-9dad-a34e029d2657 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-80cca1d5-2ec1-4ace-9dad-a34e029d2657 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:31:07.059020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:07.059926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:07.060891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:07.060891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:07.060891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:31:07.060891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:31:07.060891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:31:07.060891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:31:07.060891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:31:07.060891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:31:07.060891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:31:07.060891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:07.066956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d8d92-3890-427e-b24c-91c98ce18c45 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:07.067142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 475/942] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Aug 31 11:31:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:07.069372 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cca1d5-2ec1-4ace-9dad-a34e029d2657 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:07.140894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:07.155496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:07.173867 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d8d92-3890-427e-b24c-91c98ce18c45 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:07.174183 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cca1d5-2ec1-4ace-9dad-a34e029d2657 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:07.209871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a76d9d-5563-4696-a4d4-90b5d3d37a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:07.210613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a76d9d-5563-4696-a4d4-90b5d3d37a0e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.210766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d31149-da28-475a-b445-6f972537fdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:07.210977 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a76d9d-5563-4696-a4d4-90b5d3d37a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.211167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a76d9d-5563-4696-a4d4-90b5d3d37a0e None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.211368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d31149-da28-475a-b445-6f972537fdb3 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.211652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d31149-da28-475a-b445-6f972537fdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.211896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a76d9d-5563-4696-a4d4-90b5d3d37a0e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:07.211998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d31149-da28-475a-b445-6f972537fdb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.212220 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a76d9d-5563-4696-a4d4-90b5d3d37a0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:07.212904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d31149-da28-475a-b445-6f972537fdb3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:07.213839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a76d9d-5563-4696-a4d4-90b5d3d37a0e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:07.214323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d31149-da28-475a-b445-6f972537fdb3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:07.217939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: WARNING keystone.server.flask.common [None req-61d31149-da28-475a-b445-6f972537fdb3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 11:31:07.235980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 476/943] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Aug 31 11:31:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:07.236898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a76d9d-5563-4696-a4d4-90b5d3d37a0e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:07.238121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.sql.core [None req-61d31149-da28-475a-b445-6f972537fdb3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 11:31:07.238121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 11:31:07.238121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [parameters: {'id': 'dc8dda44a5a54aa0a0addce46e4d37ea', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 11:31:07.238121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81621) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-61d31149-da28-475a-b445-6f972537fdb3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dc8dda44a5a54aa0a0addce46e4d37ea.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dc8dda44a5a54aa0a0addce46e4d37ea. Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application conn.query(q) Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 11:31:07.255849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result.read() Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 11:31:07.256988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application next(self.gen) Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application yield resource Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application next(self.gen) Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise self.value Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application session.commit() Aug 31 11:31:07.258026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.session.flush() Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application compat.raise_( Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 11:31:07.258901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 11:31:07.260298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application conn.query(q) Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result.read() Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 11:31:07.261998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [parameters: {'id': 'dc8dda44a5a54aa0a0addce46e4d37ea', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:07.263303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:07.264535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:07.266032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:07.266032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 11:31:07.266032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 11:31:07.266032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:07.266032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:07.266032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 11:31:07.266032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 11:31:07.266032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dc8dda44a5a54aa0a0addce46e4d37ea. Aug 31 11:31:07.266032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:07.266032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 468/944] 10.222.0.60 () {60 vars in 1140 bytes} [Wed Aug 31 11:31:07 2022] POST /identity/v3/projects => generated 262 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:07.266032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d31149-da28-475a-b445-6f972537fdb3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:07.355602 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a76d9d-5563-4696-a4d4-90b5d3d37a0e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:07.368431 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d31149-da28-475a-b445-6f972537fdb3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:07.370377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:07.391928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a988a-9946-48d5-95fa-303c26add3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:07.392588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a988a-9946-48d5-95fa-303c26add3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.392875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a988a-9946-48d5-95fa-303c26add3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.393149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a988a-9946-48d5-95fa-303c26add3c6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.393738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a988a-9946-48d5-95fa-303c26add3c6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:07.395086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a988a-9946-48d5-95fa-303c26add3c6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:07.404033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648fdc1-9574-4972-8025-31628e69b870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1e2a988a-9946-48d5-95fa-303c26add3c6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:07.404466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:07.405505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:07.406341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:07.407467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fdc1-9574-4972-8025-31628e69b870 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.407467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fdc1-9574-4972-8025-31628e69b870 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.407467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fdc1-9574-4972-8025-31628e69b870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.410535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 477/945] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Aug 31 11:31:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:07.411387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a988a-9946-48d5-95fa-303c26add3c6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:07.417826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:07.421223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648fdc1-9574-4972-8025-31628e69b870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:07.423242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648fdc1-9574-4972-8025-31628e69b870 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d648fdc1-9574-4972-8025-31628e69b870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:07.440185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:07.441396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:07.442743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:07.442743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:07.442743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:07.443146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 469/946] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:07.443943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648fdc1-9574-4972-8025-31628e69b870 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:07.520865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a988a-9946-48d5-95fa-303c26add3c6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:07.547628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648fdc1-9574-4972-8025-31628e69b870 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:07.555098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2a5ce-1ec5-49c8-9f77-1e4cdcfe7c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:07.555719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2a5ce-1ec5-49c8-9f77-1e4cdcfe7c89 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.556551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2a5ce-1ec5-49c8-9f77-1e4cdcfe7c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.556830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2a5ce-1ec5-49c8-9f77-1e4cdcfe7c89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.563625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2a5ce-1ec5-49c8-9f77-1e4cdcfe7c89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:07.565090 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2a5ce-1ec5-49c8-9f77-1e4cdcfe7c89 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-64e2a5ce-1ec5-49c8-9f77-1e4cdcfe7c89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:31:07.577585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:07.578671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:07.579585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:31:07.579585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:31:07.579585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:31:07.579585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:31:07.579585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:31:07.579585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:31:07.579585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:07.579585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:07.579585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 478/947] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:31:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:07.580083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2a5ce-1ec5-49c8-9f77-1e4cdcfe7c89 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:07.586592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050d2ea-89de-4e42-8558-6086bab0aea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:07.587723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050d2ea-89de-4e42-8558-6086bab0aea6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.588018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050d2ea-89de-4e42-8558-6086bab0aea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.588297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050d2ea-89de-4e42-8558-6086bab0aea6 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.588944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050d2ea-89de-4e42-8558-6086bab0aea6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:07.589248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050d2ea-89de-4e42-8558-6086bab0aea6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:07.590614 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050d2ea-89de-4e42-8558-6086bab0aea6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:07.598611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 470/948] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Aug 31 11:31:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:07.603934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050d2ea-89de-4e42-8558-6086bab0aea6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:07.622231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:07.660983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:07.696816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2a5ce-1ec5-49c8-9f77-1e4cdcfe7c89 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:07.721271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050d2ea-89de-4e42-8558-6086bab0aea6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:07.737644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc8779-6481-4510-8ea7-f7fdb7e95f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:07.738244 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc8779-6481-4510-8ea7-f7fdb7e95f7f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.738524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc8779-6481-4510-8ea7-f7fdb7e95f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.738793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc8779-6481-4510-8ea7-f7fdb7e95f7f None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.739426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc8779-6481-4510-8ea7-f7fdb7e95f7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:07.739730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc8779-6481-4510-8ea7-f7fdb7e95f7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:07.741099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc8779-6481-4510-8ea7-f7fdb7e95f7f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:07.752183 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 479/949] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:07.752998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc8779-6481-4510-8ea7-f7fdb7e95f7f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:07.760918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b34ca-92d8-4d17-868e-61ae601c7509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:07.761523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b34ca-92d8-4d17-868e-61ae601c7509 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.761799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b34ca-92d8-4d17-868e-61ae601c7509 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.762067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b34ca-92d8-4d17-868e-61ae601c7509 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.762759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b34ca-92d8-4d17-868e-61ae601c7509 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:07.763057 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b34ca-92d8-4d17-868e-61ae601c7509 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:07.764480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b34ca-92d8-4d17-868e-61ae601c7509 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:07.782352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 471/950] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:31:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:07.783589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf272e-8c68-487d-84d0-95ab108d4b0a None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.783879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf272e-8c68-487d-84d0-95ab108d4b0a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.784155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf272e-8c68-487d-84d0-95ab108d4b0a None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.785526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 472/951] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:07.789767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf272e-8c68-487d-84d0-95ab108d4b0a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:07.856828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:07.863810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc8779-6481-4510-8ea7-f7fdb7e95f7f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:07.903035 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc0886-8f30-4fde-b279-5ae89940e120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:07.903755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc0886-8f30-4fde-b279-5ae89940e120 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.904031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc0886-8f30-4fde-b279-5ae89940e120 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.904422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc0886-8f30-4fde-b279-5ae89940e120 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.905085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc0886-8f30-4fde-b279-5ae89940e120 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:07.905386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc0886-8f30-4fde-b279-5ae89940e120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:07.907355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc0886-8f30-4fde-b279-5ae89940e120 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:07.908746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf272e-8c68-487d-84d0-95ab108d4b0a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:07.913407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 480/952] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Aug 31 11:31:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:07.914703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e2599-53b6-4f09-9b03-328291adb2df None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.914970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e2599-53b6-4f09-9b03-328291adb2df None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.915239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e2599-53b6-4f09-9b03-328291adb2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.930809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:07.934272 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-752e2599-53b6-4f09-9b03-328291adb2df None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:07.935470 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-752e2599-53b6-4f09-9b03-328291adb2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:07.954297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffd740-2347-4c20-908e-0876bdc7865b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:07.955027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffd740-2347-4c20-908e-0876bdc7865b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:07.955385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffd740-2347-4c20-908e-0876bdc7865b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:07.955767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffd740-2347-4c20-908e-0876bdc7865b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:07.965019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffd740-2347-4c20-908e-0876bdc7865b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:07.966485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffd740-2347-4c20-908e-0876bdc7865b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-30ffd740-2347-4c20-908e-0876bdc7865b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:31:07.978682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:07.979942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:07.981033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:31:07.981033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:31:07.981033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:31:07.981033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:31:07.981033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:31:07.981033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:31:07.981033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:07.981033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:07.985450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 473/953] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:31:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:07.985450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffd740-2347-4c20-908e-0876bdc7865b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:08.017641 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 481/954] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:08.018394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e2599-53b6-4f09-9b03-328291adb2df None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:08.122143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffd740-2347-4c20-908e-0876bdc7865b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:08.129032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:08.143684 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e2599-53b6-4f09-9b03-328291adb2df None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:08.160550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb34ff9-d39b-4b21-8f9c-009a55b50642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:08.161312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb34ff9-d39b-4b21-8f9c-009a55b50642 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:08.161693 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb34ff9-d39b-4b21-8f9c-009a55b50642 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:08.162068 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb34ff9-d39b-4b21-8f9c-009a55b50642 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:08.162904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb34ff9-d39b-4b21-8f9c-009a55b50642 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:08.164752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb34ff9-d39b-4b21-8f9c-009a55b50642 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:08.189314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f84a2-10c3-43de-b19e-f8b3a15a3637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:08.190126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 474/955] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:31:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:08.190498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f84a2-10c3-43de-b19e-f8b3a15a3637 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:08.190764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f84a2-10c3-43de-b19e-f8b3a15a3637 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:08.191032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f84a2-10c3-43de-b19e-f8b3a15a3637 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:08.192054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f84a2-10c3-43de-b19e-f8b3a15a3637 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:08.192362 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f84a2-10c3-43de-b19e-f8b3a15a3637 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:08.193827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f84a2-10c3-43de-b19e-f8b3a15a3637 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:08.214032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 482/956] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:31:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:08.220211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb34ff9-d39b-4b21-8f9c-009a55b50642 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:08.284596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:08.318083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f84a2-10c3-43de-b19e-f8b3a15a3637 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:08.353547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb34ff9-d39b-4b21-8f9c-009a55b50642 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:08.395997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068791bb-04d7-4e9a-819b-aeb68f0ab6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:08.397371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068791bb-04d7-4e9a-819b-aeb68f0ab6ab None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:08.397656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068791bb-04d7-4e9a-819b-aeb68f0ab6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:08.397930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068791bb-04d7-4e9a-819b-aeb68f0ab6ab None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/16216de2aa9c4a6388f94263444965df/roles/455a7f6a98b44607998cf25d8dec97cc` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:08.400875 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068791bb-04d7-4e9a-819b-aeb68f0ab6ab None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=16216de2aa9c4a6388f94263444965df, role_id=455a7f6a98b44607998cf25d8dec97cc)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:08.402658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068791bb-04d7-4e9a-819b-aeb68f0ab6ab None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:08.420539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 475/957] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:31:08 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/16216de2aa9c4a6388f94263444965df/roles/455a7f6a98b44607998cf25d8dec97cc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:31:08.445045 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f84a2-10c3-43de-b19e-f8b3a15a3637 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:08.488788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1455481-bd5d-4fab-8306-326e1c0d2948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:08.489556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1455481-bd5d-4fab-8306-326e1c0d2948 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:08.489840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1455481-bd5d-4fab-8306-326e1c0d2948 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:08.490209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1455481-bd5d-4fab-8306-326e1c0d2948 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:08.490973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1455481-bd5d-4fab-8306-326e1c0d2948 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:08.492565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1455481-bd5d-4fab-8306-326e1c0d2948 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b1455481-bd5d-4fab-8306-326e1c0d2948 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:08.504749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:08.505884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:08.506865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 483/958] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:31:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:08.514326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068791bb-04d7-4e9a-819b-aeb68f0ab6ab None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:08.613949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:08.628970 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068791bb-04d7-4e9a-819b-aeb68f0ab6ab None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:08.664132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e99ae-96db-478e-8291-670188dfb0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:08.665040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e99ae-96db-478e-8291-670188dfb0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:08.666154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e99ae-96db-478e-8291-670188dfb0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:08.666606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e99ae-96db-478e-8291-670188dfb0c5 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:08.667408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e99ae-96db-478e-8291-670188dfb0c5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:08.667918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e99ae-96db-478e-8291-670188dfb0c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:08.669514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e99ae-96db-478e-8291-670188dfb0c5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:08.675197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 476/959] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:31:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:08.681078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1455481-bd5d-4fab-8306-326e1c0d2948 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:08.700345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:08.792216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1455481-bd5d-4fab-8306-326e1c0d2948 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:08.830532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57f27b-c274-4a3c-a749-d0115050821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:08.831491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57f27b-c274-4a3c-a749-d0115050821e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:08.831950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57f27b-c274-4a3c-a749-d0115050821e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:08.832421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57f27b-c274-4a3c-a749-d0115050821e None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:08.833266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57f27b-c274-4a3c-a749-d0115050821e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:08.833741 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57f27b-c274-4a3c-a749-d0115050821e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:08.835330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57f27b-c274-4a3c-a749-d0115050821e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:08.841184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 484/960] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:31:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:09.807588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e55b11-fdbe-4fac-a45e-dadaac83d4f4 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:09.808114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e55b11-fdbe-4fac-a45e-dadaac83d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:09.808114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e55b11-fdbe-4fac-a45e-dadaac83d4f4 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:09.809835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 477/961] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:09.815214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f799d30-d9c5-44ff-bb5e-0593d1e6922f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:09.815664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f799d30-d9c5-44ff-bb5e-0593d1e6922f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:09.815745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f799d30-d9c5-44ff-bb5e-0593d1e6922f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:09.835068 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8f799d30-d9c5-44ff-bb5e-0593d1e6922f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:09.836266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8f799d30-d9c5-44ff-bb5e-0593d1e6922f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:09.909816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 485/962] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:10.186962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e55b11-fdbe-4fac-a45e-dadaac83d4f4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:10.251828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601501e2-6805-477a-b8f0-f1f7fc3fac4e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:10.252158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601501e2-6805-477a-b8f0-f1f7fc3fac4e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:10.252392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601501e2-6805-477a-b8f0-f1f7fc3fac4e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:10.254007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 486/963] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:10.259002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9cc8a-2789-4ba4-a997-adf5d6064af5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:10.259454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9cc8a-2789-4ba4-a997-adf5d6064af5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:10.259738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9cc8a-2789-4ba4-a997-adf5d6064af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:10.278415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-08d9cc8a-2789-4ba4-a997-adf5d6064af5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:10.279718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-08d9cc8a-2789-4ba4-a997-adf5d6064af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:10.281987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:10.299760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e55b11-fdbe-4fac-a45e-dadaac83d4f4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:10.337859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ade6fd-e57b-4518-ab8f-8904dddd737f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:10.338477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ade6fd-e57b-4518-ab8f-8904dddd737f None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:10.338754 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ade6fd-e57b-4518-ab8f-8904dddd737f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:10.339028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ade6fd-e57b-4518-ab8f-8904dddd737f None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:10.339679 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ade6fd-e57b-4518-ab8f-8904dddd737f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:10.341269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ade6fd-e57b-4518-ab8f-8904dddd737f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:10.358299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.sql.core [None req-b8ade6fd-e57b-4518-ab8f-8904dddd737f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 11:31:10.358299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 11:31:10.358299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [parameters: {'user_id': '5adf3e5e0d9a42ddb10ce85832c89a92', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 11:31:10.358299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81621) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b8ade6fd-e57b-4518-ab8f-8904dddd737f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application conn.query(q) Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result.read() Aug 31 11:31:10.367799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 11:31:10.368803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application next(self.gen) Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application yield resource Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application next(self.gen) Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise self.value Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application session.commit() Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 11:31:10.369745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.session.flush() Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application compat.raise_( Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 11:31:10.370629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 11:31:10.371514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application conn.query(q) Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result.read() Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 11:31:10.372376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [parameters: {'user_id': '5adf3e5e0d9a42ddb10ce85832c89a92', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:10.376836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 11:31:10.377647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 11:31:10.378543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 11:31:10.378543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 11:31:10.378543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 11:31:10.378543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 11:31:10.378543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 11:31:10.378543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:10.378543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 478/965] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Aug 31 11:31:10 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:10.378983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 487/965] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:10.379069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ade6fd-e57b-4518-ab8f-8904dddd737f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:10.396768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:10.441664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0e73-8f4b-4e9b-8410-917d7ce0590e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:10.442657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0e73-8f4b-4e9b-8410-917d7ce0590e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:10.443024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0e73-8f4b-4e9b-8410-917d7ce0590e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:10.445271 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 488/966] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:31:10.450398 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84659a-1a76-4df0-8073-0e9ef2f4982d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:10.450924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84659a-1a76-4df0-8073-0e9ef2f4982d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:10.451327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84659a-1a76-4df0-8073-0e9ef2f4982d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:10.478096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ed84659a-1a76-4df0-8073-0e9ef2f4982d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:10.480168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ed84659a-1a76-4df0-8073-0e9ef2f4982d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:10.496529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ade6fd-e57b-4518-ab8f-8904dddd737f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:10.534810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66680cde-e3d6-4bff-8a3b-01bec6235b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:10.535444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680cde-e3d6-4bff-8a3b-01bec6235b01 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:10.535727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680cde-e3d6-4bff-8a3b-01bec6235b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:10.536007 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680cde-e3d6-4bff-8a3b-01bec6235b01 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:10.544955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66680cde-e3d6-4bff-8a3b-01bec6235b01 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:10.547084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66680cde-e3d6-4bff-8a3b-01bec6235b01 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-66680cde-e3d6-4bff-8a3b-01bec6235b01 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:31:10.559827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:10.560973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:10.562097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:10.562097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:10.562097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:31:10.562097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:31:10.562097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:31:10.562097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:31:10.562097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:31:10.562097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:31:10.562097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 11:31:10.562097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:10.565868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 479/967] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:31:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:10.566956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66680cde-e3d6-4bff-8a3b-01bec6235b01 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:10.578524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 489/968] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:10.647809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84659a-1a76-4df0-8073-0e9ef2f4982d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:10.652656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:10.690114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66680cde-e3d6-4bff-8a3b-01bec6235b01 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:10.726900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:10.743923 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d993758-a947-45fb-ad88-c146854e6fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:10.745270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993758-a947-45fb-ad88-c146854e6fed None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:10.746201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993758-a947-45fb-ad88-c146854e6fed None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:10.746937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993758-a947-45fb-ad88-c146854e6fed None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:10.747853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d993758-a947-45fb-ad88-c146854e6fed None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:10.748602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d993758-a947-45fb-ad88-c146854e6fed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:10.750636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d993758-a947-45fb-ad88-c146854e6fed None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:10.765448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84659a-1a76-4df0-8073-0e9ef2f4982d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:10.802905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91797a-55bd-4337-9993-c563c4c09b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:10.803507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 480/969] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Aug 31 11:31:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:10.803619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91797a-55bd-4337-9993-c563c4c09b74 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:10.804038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91797a-55bd-4337-9993-c563c4c09b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:10.804384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91797a-55bd-4337-9993-c563c4c09b74 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:10.805073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91797a-55bd-4337-9993-c563c4c09b74 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:10.806753 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91797a-55bd-4337-9993-c563c4c09b74 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-bc91797a-55bd-4337-9993-c563c4c09b74 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:10.821497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:10.824475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:10.826012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 490/970] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Aug 31 11:31:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:10.831829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d993758-a947-45fb-ad88-c146854e6fed None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:10.841325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1e9bf-78ce-47d3-a578-2c11c477ba78 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:10.841595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1e9bf-78ce-47d3-a578-2c11c477ba78 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:10.841860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1e9bf-78ce-47d3-a578-2c11c477ba78 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:10.843487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 491/971] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:10.848915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6094ed4-c29a-401f-a596-d23dafaaae7a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:10.849206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6094ed4-c29a-401f-a596-d23dafaaae7a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:10.849480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6094ed4-c29a-401f-a596-d23dafaaae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:10.868810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f6094ed4-c29a-401f-a596-d23dafaaae7a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:10.870032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f6094ed4-c29a-401f-a596-d23dafaaae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:10.957466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 492/972] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:10.958691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6094ed4-c29a-401f-a596-d23dafaaae7a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:10.967279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d993758-a947-45fb-ad88-c146854e6fed None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:11.003355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:11.011326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bba33b-40db-4967-ad4c-e5e3a31b399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:11.012072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bba33b-40db-4967-ad4c-e5e3a31b399a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:11.012376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bba33b-40db-4967-ad4c-e5e3a31b399a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:11.012691 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bba33b-40db-4967-ad4c-e5e3a31b399a None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:11.013338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bba33b-40db-4967-ad4c-e5e3a31b399a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:11.013642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bba33b-40db-4967-ad4c-e5e3a31b399a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:11.015022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bba33b-40db-4967-ad4c-e5e3a31b399a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:11.020317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 481/973] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Aug 31 11:31:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:11.027299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bba33b-40db-4967-ad4c-e5e3a31b399a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:11.043225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:11.108939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6094ed4-c29a-401f-a596-d23dafaaae7a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:11.160567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717352f1-e0f2-4e98-aa72-0979ea44f06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:11.161260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717352f1-e0f2-4e98-aa72-0979ea44f06a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:11.161542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717352f1-e0f2-4e98-aa72-0979ea44f06a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:11.161811 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717352f1-e0f2-4e98-aa72-0979ea44f06a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:11.171931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bba33b-40db-4967-ad4c-e5e3a31b399a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:11.173676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717352f1-e0f2-4e98-aa72-0979ea44f06a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-717352f1-e0f2-4e98-aa72-0979ea44f06a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:11.177710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:11.178750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:11.185500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 493/974] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Aug 31 11:31:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:11.187656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717352f1-e0f2-4e98-aa72-0979ea44f06a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:11.223988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df6417-f925-4f9a-a5f2-15e68853a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:11.224637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df6417-f925-4f9a-a5f2-15e68853a58f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:11.224924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df6417-f925-4f9a-a5f2-15e68853a58f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:11.225203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df6417-f925-4f9a-a5f2-15e68853a58f None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:11.226654 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df6417-f925-4f9a-a5f2-15e68853a58f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:11.227150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df6417-f925-4f9a-a5f2-15e68853a58f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:11.228634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df6417-f925-4f9a-a5f2-15e68853a58f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:11.235024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 482/975] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Aug 31 11:31:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:11.235878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df6417-f925-4f9a-a5f2-15e68853a58f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:11.307745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:11.317556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717352f1-e0f2-4e98-aa72-0979ea44f06a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:11.347969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df6417-f925-4f9a-a5f2-15e68853a58f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:11.348255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:11.363645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6540de-3863-4e7c-809f-03928d5a0b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:11.364253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6540de-3863-4e7c-809f-03928d5a0b0e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:11.364555 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6540de-3863-4e7c-809f-03928d5a0b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:11.364907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6540de-3863-4e7c-809f-03928d5a0b0e None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:11.365590 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6540de-3863-4e7c-809f-03928d5a0b0e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:11.365892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6540de-3863-4e7c-809f-03928d5a0b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:11.368009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6540de-3863-4e7c-809f-03928d5a0b0e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:11.378378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 494/976] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Aug 31 11:31:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:11.378378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6540de-3863-4e7c-809f-03928d5a0b0e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:11.388454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc82943-f77a-4a1d-b948-af65326e11de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:11.389087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc82943-f77a-4a1d-b948-af65326e11de None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:11.389371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc82943-f77a-4a1d-b948-af65326e11de None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:11.389636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc82943-f77a-4a1d-b948-af65326e11de None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:11.390287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc82943-f77a-4a1d-b948-af65326e11de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:11.391658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc82943-f77a-4a1d-b948-af65326e11de None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:11.407178 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 483/977] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:31:11 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:11.408046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc82943-f77a-4a1d-b948-af65326e11de None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:11.521463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6540de-3863-4e7c-809f-03928d5a0b0e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:11.544366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc82943-f77a-4a1d-b948-af65326e11de None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:11.563463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:11.580912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069cb83-0e07-4fc0-891f-68265566c8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:11.581531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069cb83-0e07-4fc0-891f-68265566c8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:11.581808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069cb83-0e07-4fc0-891f-68265566c8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:11.582076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069cb83-0e07-4fc0-891f-68265566c8a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:11.582800 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069cb83-0e07-4fc0-891f-68265566c8a3 None admin] RBAC: Adding query filter params (interface=admin, service_id=08fcfb8bc61c46309434b7f83c40be79, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:11.583106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069cb83-0e07-4fc0-891f-68265566c8a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:11.584483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069cb83-0e07-4fc0-891f-68265566c8a3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:11.596259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 495/978] 10.222.0.60 () {56 vars in 1249 bytes} [Wed Aug 31 11:31:11 2022] GET /identity/v3/endpoints?service_id=08fcfb8bc61c46309434b7f83c40be79&interface=admin®ion_id=RegionOne => generated 193 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:11.598234 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069cb83-0e07-4fc0-891f-68265566c8a3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:11.600280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00bd7c-47e4-495d-9c0e-a6c02e754504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:11.600965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00bd7c-47e4-495d-9c0e-a6c02e754504 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:11.601246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00bd7c-47e4-495d-9c0e-a6c02e754504 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:11.601519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00bd7c-47e4-495d-9c0e-a6c02e754504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:11.613453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00bd7c-47e4-495d-9c0e-a6c02e754504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:11.614930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00bd7c-47e4-495d-9c0e-a6c02e754504 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-cd00bd7c-47e4-495d-9c0e-a6c02e754504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:11.635093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:11.636270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:11.637808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:11.637808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:11.637808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:11.637808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 484/979] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:11.642531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00bd7c-47e4-495d-9c0e-a6c02e754504 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:11.668536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:11.736711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069cb83-0e07-4fc0-891f-68265566c8a3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:11.781187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0daaff8-ea6f-48d7-83e9-bb54198dd783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:11.781801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daaff8-ea6f-48d7-83e9-bb54198dd783 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:11.782080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daaff8-ea6f-48d7-83e9-bb54198dd783 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:11.782335 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daaff8-ea6f-48d7-83e9-bb54198dd783 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:11.782951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0daaff8-ea6f-48d7-83e9-bb54198dd783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:11.784349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0daaff8-ea6f-48d7-83e9-bb54198dd783 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:11.792273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 496/980] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Aug 31 11:31:11 2022] GET /identity/v3/services => generated 1727 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:31:11.800908 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00bd7c-47e4-495d-9c0e-a6c02e754504 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:11.857360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d19e-9b23-4150-90d6-7363c1466eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:11.857992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d19e-9b23-4150-90d6-7363c1466eac None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:11.858244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d19e-9b23-4150-90d6-7363c1466eac None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:11.858446 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d19e-9b23-4150-90d6-7363c1466eac None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:11.860032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4d19e-9b23-4150-90d6-7363c1466eac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:11.860351 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4d19e-9b23-4150-90d6-7363c1466eac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:11.867316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4d19e-9b23-4150-90d6-7363c1466eac None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:11.891413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 485/981] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:11.897038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0daaff8-ea6f-48d7-83e9-bb54198dd783 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:12.018522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0daaff8-ea6f-48d7-83e9-bb54198dd783 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:12.032346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:12.055511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01291f-8704-4f85-9f7f-0aa97be5a8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:12.056319 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01291f-8704-4f85-9f7f-0aa97be5a8eb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:12.056777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01291f-8704-4f85-9f7f-0aa97be5a8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:12.057053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01291f-8704-4f85-9f7f-0aa97be5a8eb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:12.064931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01291f-8704-4f85-9f7f-0aa97be5a8eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:12.066357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01291f-8704-4f85-9f7f-0aa97be5a8eb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1c01291f-8704-4f85-9f7f-0aa97be5a8eb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:31:12.077793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:12.078740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:12.079700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:12.079700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:12.079700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:31:12.079700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:31:12.079700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:31:12.079700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:31:12.079700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:31:12.079700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:31:12.079700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 11:31:12.079700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:12.079700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 497/982] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Aug 31 11:31:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:12.086607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d19e-9b23-4150-90d6-7363c1466eac None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:12.117849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:12.202038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d19e-9b23-4150-90d6-7363c1466eac None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:12.237417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22aba8-9f3b-4d3a-9335-2a8404926528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:12.238287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22aba8-9f3b-4d3a-9335-2a8404926528 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:12.238749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22aba8-9f3b-4d3a-9335-2a8404926528 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:12.239209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22aba8-9f3b-4d3a-9335-2a8404926528 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:12.240138 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22aba8-9f3b-4d3a-9335-2a8404926528 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:12.240665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22aba8-9f3b-4d3a-9335-2a8404926528 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:12.242270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22aba8-9f3b-4d3a-9335-2a8404926528 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:12.264730 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 486/983] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Aug 31 11:31:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:12.266546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01291f-8704-4f85-9f7f-0aa97be5a8eb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:12.374783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01291f-8704-4f85-9f7f-0aa97be5a8eb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:12.409894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e04f3-6885-419f-ab08-213bd484ab53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:12.410587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e04f3-6885-419f-ab08-213bd484ab53 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:12.410872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e04f3-6885-419f-ab08-213bd484ab53 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:12.411147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e04f3-6885-419f-ab08-213bd484ab53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:12.452574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:12.481777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e04f3-6885-419f-ab08-213bd484ab53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:12.483230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e04f3-6885-419f-ab08-213bd484ab53 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:12.586893 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 498/984] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:12 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:31:12.589112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22aba8-9f3b-4d3a-9335-2a8404926528 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:12.676165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:12.699344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22aba8-9f3b-4d3a-9335-2a8404926528 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:12.733661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e8748-cc35-40e9-90a6-b98b2fea2ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:12.734289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e8748-cc35-40e9-90a6-b98b2fea2ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:12.734569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e8748-cc35-40e9-90a6-b98b2fea2ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:12.734842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e8748-cc35-40e9-90a6-b98b2fea2ce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:12.745716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e8748-cc35-40e9-90a6-b98b2fea2ce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:12.747161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e8748-cc35-40e9-90a6-b98b2fea2ce8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b31e8748-cc35-40e9-90a6-b98b2fea2ce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:12.762448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:12.763370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:12.764257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:12.764257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:12.764257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:12.764257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 487/985] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:12.769127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e04f3-6885-419f-ab08-213bd484ab53 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:12.840751 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:12.872873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e04f3-6885-419f-ab08-213bd484ab53 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:12.907857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18401c0c-098d-48c9-85f8-0c652256c256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:12.908692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18401c0c-098d-48c9-85f8-0c652256c256 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:12.908787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18401c0c-098d-48c9-85f8-0c652256c256 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:12.909027 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18401c0c-098d-48c9-85f8-0c652256c256 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:12.909666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18401c0c-098d-48c9-85f8-0c652256c256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:12.909966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18401c0c-098d-48c9-85f8-0c652256c256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:12.911896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18401c0c-098d-48c9-85f8-0c652256c256 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:12.929566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 499/986] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:12.930662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e8748-cc35-40e9-90a6-b98b2fea2ce8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:13.033399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e8748-cc35-40e9-90a6-b98b2fea2ce8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:13.066497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886ee48-facb-472d-812b-d73a006b44b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:13.067616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886ee48-facb-472d-812b-d73a006b44b2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:13.068042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886ee48-facb-472d-812b-d73a006b44b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:13.068460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886ee48-facb-472d-812b-d73a006b44b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:13.075979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886ee48-facb-472d-812b-d73a006b44b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:13.077595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886ee48-facb-472d-812b-d73a006b44b2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:13.079454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-5886ee48-facb-472d-812b-d73a006b44b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:31:13.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:13.091607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:13.092745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:31:13.092745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:31:13.092745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:31:13.092745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:31:13.092745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:31:13.092745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:31:13.092745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:13.092745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:13.099183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 488/987] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:31:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:13.100263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18401c0c-098d-48c9-85f8-0c652256c256 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:13.204986 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18401c0c-098d-48c9-85f8-0c652256c256 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:13.234293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:13.240561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05035c38-80a0-497f-b565-c00f5969c5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:13.241165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05035c38-80a0-497f-b565-c00f5969c5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:13.241443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05035c38-80a0-497f-b565-c00f5969c5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:13.241714 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05035c38-80a0-497f-b565-c00f5969c5c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:13.242393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05035c38-80a0-497f-b565-c00f5969c5c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:13.242701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05035c38-80a0-497f-b565-c00f5969c5c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:13.244689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05035c38-80a0-497f-b565-c00f5969c5c6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:13.256780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 500/988] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:31:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:13.262696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886ee48-facb-472d-812b-d73a006b44b2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:13.363709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886ee48-facb-472d-812b-d73a006b44b2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:13.397001 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8b678-ed74-4e05-abb0-603d859888a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:13.397597 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8b678-ed74-4e05-abb0-603d859888a8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:13.397878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8b678-ed74-4e05-abb0-603d859888a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:13.398151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8b678-ed74-4e05-abb0-603d859888a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:13.398874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8b678-ed74-4e05-abb0-603d859888a8 None admin] RBAC: Adding query filter params (role.id=455a7f6a98b44607998cf25d8dec97cc, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=16216de2aa9c4a6388f94263444965df) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:13.399185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8b678-ed74-4e05-abb0-603d859888a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:13.401521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8b678-ed74-4e05-abb0-603d859888a8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:13.407835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 489/989] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:31:13 2022] GET /identity/v3/role_assignments?user.id=16216de2aa9c4a6388f94263444965df&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=455a7f6a98b44607998cf25d8dec97cc => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:13.458829 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad593875-10aa-4c88-b061-27172e462cd5 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:13.459117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad593875-10aa-4c88-b061-27172e462cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:13.459302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad593875-10aa-4c88-b061-27172e462cd5 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:13.460894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 501/990] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:13.467616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca219ef-98d3-4a6f-aa35-fd08ca30580f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:13.468126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca219ef-98d3-4a6f-aa35-fd08ca30580f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:13.468632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca219ef-98d3-4a6f-aa35-fd08ca30580f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:13.490295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cca219ef-98d3-4a6f-aa35-fd08ca30580f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:13.491890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cca219ef-98d3-4a6f-aa35-fd08ca30580f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:13.554521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:13.568161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 490/991] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:13.764034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f77d58-46c5-4ba0-b477-06bc6a433c04 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:13.764330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f77d58-46c5-4ba0-b477-06bc6a433c04 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:13.764550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f77d58-46c5-4ba0-b477-06bc6a433c04 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:13.765980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 502/992] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:13.771031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91315bb-6539-4692-ab72-2bbec9bce748 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:13.771321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91315bb-6539-4692-ab72-2bbec9bce748 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:13.771598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91315bb-6539-4692-ab72-2bbec9bce748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:13.789484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e91315bb-6539-4692-ab72-2bbec9bce748 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:13.791182 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e91315bb-6539-4692-ab72-2bbec9bce748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:13.835926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f77d58-46c5-4ba0-b477-06bc6a433c04 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:13.859916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 491/993] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:13.939932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f77d58-46c5-4ba0-b477-06bc6a433c04 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:13.974005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3077c6-ef7a-49a8-8ae0-1518d290817a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:13.974623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3077c6-ef7a-49a8-8ae0-1518d290817a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:13.974897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3077c6-ef7a-49a8-8ae0-1518d290817a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:13.975164 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3077c6-ef7a-49a8-8ae0-1518d290817a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:13.975738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3077c6-ef7a-49a8-8ae0-1518d290817a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:13.977226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3077c6-ef7a-49a8-8ae0-1518d290817a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6b3077c6-ef7a-49a8-8ae0-1518d290817a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:13.988895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:13.989974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:13.990834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 503/994] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:31:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:13.996543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91315bb-6539-4692-ab72-2bbec9bce748 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:14.098290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91315bb-6539-4692-ab72-2bbec9bce748 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:14.121339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:14.133566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41e7f7-ec68-41eb-854b-a8db5ac2a41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:14.134297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41e7f7-ec68-41eb-854b-a8db5ac2a41d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:14.134580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41e7f7-ec68-41eb-854b-a8db5ac2a41d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:14.134853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41e7f7-ec68-41eb-854b-a8db5ac2a41d None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:14.135484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41e7f7-ec68-41eb-854b-a8db5ac2a41d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:14.135788 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41e7f7-ec68-41eb-854b-a8db5ac2a41d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:14.137199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41e7f7-ec68-41eb-854b-a8db5ac2a41d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:14.141595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3077c6-ef7a-49a8-8ae0-1518d290817a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:14.142277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 492/995] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:31:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:14.148794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41e7f7-ec68-41eb-854b-a8db5ac2a41d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:14.213218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:14.255729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41e7f7-ec68-41eb-854b-a8db5ac2a41d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:14.269183 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3077c6-ef7a-49a8-8ae0-1518d290817a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:14.294231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481933f-1f95-442c-b9ca-08fc5865d668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:14.294969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481933f-1f95-442c-b9ca-08fc5865d668 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:14.295250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481933f-1f95-442c-b9ca-08fc5865d668 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:14.295525 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481933f-1f95-442c-b9ca-08fc5865d668 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:14.296149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481933f-1f95-442c-b9ca-08fc5865d668 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:14.296451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481933f-1f95-442c-b9ca-08fc5865d668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:14.297839 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481933f-1f95-442c-b9ca-08fc5865d668 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:14.303013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 493/996] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:31:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:14.309237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481933f-1f95-442c-b9ca-08fc5865d668 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:14.311206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535cb66-9580-44bb-827e-a4f37348dbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:14.311838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535cb66-9580-44bb-827e-a4f37348dbca None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:14.312119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535cb66-9580-44bb-827e-a4f37348dbca None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:14.312391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535cb66-9580-44bb-827e-a4f37348dbca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:14.312998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535cb66-9580-44bb-827e-a4f37348dbca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:14.314343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535cb66-9580-44bb-827e-a4f37348dbca None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f535cb66-9580-44bb-827e-a4f37348dbca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:14.326495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:14.327506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:14.328376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 504/997] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Aug 31 11:31:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:14.333367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535cb66-9580-44bb-827e-a4f37348dbca None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:14.366542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:14.422009 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481933f-1f95-442c-b9ca-08fc5865d668 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:14.442067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535cb66-9580-44bb-827e-a4f37348dbca None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:14.458410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54484a8-f9f2-46d1-9f61-2e65dea02c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:14.459019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54484a8-f9f2-46d1-9f61-2e65dea02c4e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:14.459301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54484a8-f9f2-46d1-9f61-2e65dea02c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:14.459577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54484a8-f9f2-46d1-9f61-2e65dea02c4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:14.460400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54484a8-f9f2-46d1-9f61-2e65dea02c4e None admin] RBAC: Adding query filter params (interface=public, service_id=c48bc3d6e6064a739484fd6c05b0a2a5, region_id=RegionOne) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:14.460856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54484a8-f9f2-46d1-9f61-2e65dea02c4e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:14.462776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54484a8-f9f2-46d1-9f61-2e65dea02c4e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:14.464634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:14.469855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 494/998] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Aug 31 11:31:14 2022] GET /identity/v3/endpoints?service_id=c48bc3d6e6064a739484fd6c05b0a2a5&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:14.474979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54484a8-f9f2-46d1-9f61-2e65dea02c4e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:14.484655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe5881-031f-49b8-891c-e4e22ad3896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:14.485271 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe5881-031f-49b8-891c-e4e22ad3896e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:14.485556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe5881-031f-49b8-891c-e4e22ad3896e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:14.485858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe5881-031f-49b8-891c-e4e22ad3896e None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:14.486494 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe5881-031f-49b8-891c-e4e22ad3896e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:14.486797 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe5881-031f-49b8-891c-e4e22ad3896e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:14.488151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe5881-031f-49b8-891c-e4e22ad3896e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:14.493004 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 505/999] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Aug 31 11:31:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:14.498692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe5881-031f-49b8-891c-e4e22ad3896e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:14.586264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54484a8-f9f2-46d1-9f61-2e65dea02c4e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:14.600815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:14.613285 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe5881-031f-49b8-891c-e4e22ad3896e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:14.622052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7c06e-864a-4247-a830-0fadcf0e5bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:14.622603 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7c06e-864a-4247-a830-0fadcf0e5bcf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:14.622887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7c06e-864a-4247-a830-0fadcf0e5bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:14.623167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7c06e-864a-4247-a830-0fadcf0e5bcf None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:14.623759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7c06e-864a-4247-a830-0fadcf0e5bcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:14.625153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7c06e-864a-4247-a830-0fadcf0e5bcf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:14.631369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 495/1000] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Aug 31 11:31:14 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:31:14.656774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36619d30-bd9a-4831-951e-9860aa647ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:14.657741 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36619d30-bd9a-4831-951e-9860aa647ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:14.658267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36619d30-bd9a-4831-951e-9860aa647ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:14.658748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36619d30-bd9a-4831-951e-9860aa647ad1 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:14.659585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36619d30-bd9a-4831-951e-9860aa647ad1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:14.660132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36619d30-bd9a-4831-951e-9860aa647ad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:14.661802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36619d30-bd9a-4831-951e-9860aa647ad1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:14.668969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 506/1001] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Aug 31 11:31:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:14.677095 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7c06e-864a-4247-a830-0fadcf0e5bcf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:14.789948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7c06e-864a-4247-a830-0fadcf0e5bcf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:14.826448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e1d0e-eca3-40e3-a766-0942627bea76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:14.827059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e1d0e-eca3-40e3-a766-0942627bea76 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:14.827344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e1d0e-eca3-40e3-a766-0942627bea76 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:14.827620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e1d0e-eca3-40e3-a766-0942627bea76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:14.828303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e1d0e-eca3-40e3-a766-0942627bea76 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:14.829717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e1d0e-eca3-40e3-a766-0942627bea76 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:14.846401 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 496/1002] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:31:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:15.372973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82d5f-6105-47d0-b24e-867606ac6a4a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:15.373497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82d5f-6105-47d0-b24e-867606ac6a4a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:15.373575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82d5f-6105-47d0-b24e-867606ac6a4a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:15.375155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 507/1003] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:15.380074 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115bc75-d430-4ab8-af49-25a9481adfca None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:15.380473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115bc75-d430-4ab8-af49-25a9481adfca None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:15.380673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115bc75-d430-4ab8-af49-25a9481adfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:15.399791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5115bc75-d430-4ab8-af49-25a9481adfca None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:15.401376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5115bc75-d430-4ab8-af49-25a9481adfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:15.433175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:15.480541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 497/1004] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:15.756634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c82d5f-6105-47d0-b24e-867606ac6a4a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:15.814896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:15.870015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c82d5f-6105-47d0-b24e-867606ac6a4a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:15.904181 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c634770-dd36-4b5b-95d4-89a385771af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:15.904834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c634770-dd36-4b5b-95d4-89a385771af8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:15.905111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c634770-dd36-4b5b-95d4-89a385771af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:15.905378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c634770-dd36-4b5b-95d4-89a385771af8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:15.905950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c634770-dd36-4b5b-95d4-89a385771af8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:15.907308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c634770-dd36-4b5b-95d4-89a385771af8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9c634770-dd36-4b5b-95d4-89a385771af8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:15.916105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:15.917489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:15.918349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 508/1005] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Aug 31 11:31:15 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:15.925065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115bc75-d430-4ab8-af49-25a9481adfca None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:16.028641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115bc75-d430-4ab8-af49-25a9481adfca None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:16.062458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaafbb4-7d4e-4758-a1e6-13984a9b4298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:16.063070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaafbb4-7d4e-4758-a1e6-13984a9b4298 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:16.063352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaafbb4-7d4e-4758-a1e6-13984a9b4298 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:16.063625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaafbb4-7d4e-4758-a1e6-13984a9b4298 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:16.064254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaafbb4-7d4e-4758-a1e6-13984a9b4298 None admin] RBAC: Adding query filter params (name=image) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:16.064582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaafbb4-7d4e-4758-a1e6-13984a9b4298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:16.065947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaafbb4-7d4e-4758-a1e6-13984a9b4298 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:16.070877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 498/1006] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:16.077445 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c634770-dd36-4b5b-95d4-89a385771af8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:16.180117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c634770-dd36-4b5b-95d4-89a385771af8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:16.194491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:16.217614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f70b99-cb72-4ad8-ba21-473091b1b0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:16.218266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f70b99-cb72-4ad8-ba21-473091b1b0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:16.218547 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f70b99-cb72-4ad8-ba21-473091b1b0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:16.218815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f70b99-cb72-4ad8-ba21-473091b1b0a3 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:16.219432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f70b99-cb72-4ad8-ba21-473091b1b0a3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:16.219730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f70b99-cb72-4ad8-ba21-473091b1b0a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:16.221104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f70b99-cb72-4ad8-ba21-473091b1b0a3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:16.232566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 509/1007] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:16.571131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f76f1e-c02e-435e-ad5d-f138f33e1301 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:16.571814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f76f1e-c02e-435e-ad5d-f138f33e1301 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:16.571814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f76f1e-c02e-435e-ad5d-f138f33e1301 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:16.573477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 499/1008] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:16.578311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb9586-5023-4b13-bf1c-c65d76a224a1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:16.578716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb9586-5023-4b13-bf1c-c65d76a224a1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:16.578881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb9586-5023-4b13-bf1c-c65d76a224a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:16.597420 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f6eb9586-5023-4b13-bf1c-c65d76a224a1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:16.598665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f6eb9586-5023-4b13-bf1c-c65d76a224a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:16.669085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 510/1009] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:16 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:16.776323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1de439-51b7-41a1-8db9-bb47e4a395b3 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:16.776731 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1de439-51b7-41a1-8db9-bb47e4a395b3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:16.776904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1de439-51b7-41a1-8db9-bb47e4a395b3 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:16.778398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 500/1010] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:16.783273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309652e-d3b5-4e9e-b6f2-32c65ccc4b9c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:16.783562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309652e-d3b5-4e9e-b6f2-32c65ccc4b9c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:16.783833 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309652e-d3b5-4e9e-b6f2-32c65ccc4b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:16.801993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a309652e-d3b5-4e9e-b6f2-32c65ccc4b9c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:16.803168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a309652e-d3b5-4e9e-b6f2-32c65ccc4b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:16.872895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 511/1011] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:16 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:16.947557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1de439-51b7-41a1-8db9-bb47e4a395b3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:16.977162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:17.055698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1de439-51b7-41a1-8db9-bb47e4a395b3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:17.104208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8cd5f-fdae-4581-ab73-bec878aecae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:17.104868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8cd5f-fdae-4581-ab73-bec878aecae9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:17.105153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8cd5f-fdae-4581-ab73-bec878aecae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:17.105430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8cd5f-fdae-4581-ab73-bec878aecae9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:17.106015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8cd5f-fdae-4581-ab73-bec878aecae9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:17.107393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8cd5f-fdae-4581-ab73-bec878aecae9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-e6a8cd5f-fdae-4581-ab73-bec878aecae9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:17.116231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:17.120770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:17.121676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 501/1012] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:31:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:17.123959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309652e-d3b5-4e9e-b6f2-32c65ccc4b9c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:17.141261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8cd5f-fdae-4581-ab73-bec878aecae9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:17.164191 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:17.238880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309652e-d3b5-4e9e-b6f2-32c65ccc4b9c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:17.244949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:17.255358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8cd5f-fdae-4581-ab73-bec878aecae9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:17.276810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1f423-5400-4030-a0c4-400ccecf9ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:17.277421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1f423-5400-4030-a0c4-400ccecf9ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:17.278146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1f423-5400-4030-a0c4-400ccecf9ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:17.278431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1f423-5400-4030-a0c4-400ccecf9ea1 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:17.279071 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1f423-5400-4030-a0c4-400ccecf9ea1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:17.279376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1f423-5400-4030-a0c4-400ccecf9ea1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:17.280810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1f423-5400-4030-a0c4-400ccecf9ea1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:17.292161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 512/1013] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:31:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:17.293743 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1f423-5400-4030-a0c4-400ccecf9ea1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:17.293999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c56ef-9813-4de2-812a-8c0b5226762d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:17.294653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c56ef-9813-4de2-812a-8c0b5226762d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:17.294935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c56ef-9813-4de2-812a-8c0b5226762d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:17.295211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c56ef-9813-4de2-812a-8c0b5226762d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:17.296355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c56ef-9813-4de2-812a-8c0b5226762d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:17.297871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c56ef-9813-4de2-812a-8c0b5226762d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:17.299454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 502/1014] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:17.304976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c56ef-9813-4de2-812a-8c0b5226762d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:17.402677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:17.406456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1f423-5400-4030-a0c4-400ccecf9ea1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:17.407954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c56ef-9813-4de2-812a-8c0b5226762d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:17.442849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19065ba6-bc68-41a9-b6ff-3fdc1435a262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:17.443193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3913149-9464-48ad-85c3-5f3b1d437b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:17.443512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19065ba6-bc68-41a9-b6ff-3fdc1435a262 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:17.443795 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19065ba6-bc68-41a9-b6ff-3fdc1435a262 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:17.443980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3913149-9464-48ad-85c3-5f3b1d437b37 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:17.444079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19065ba6-bc68-41a9-b6ff-3fdc1435a262 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:17.444365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3913149-9464-48ad-85c3-5f3b1d437b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:17.444738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19065ba6-bc68-41a9-b6ff-3fdc1435a262 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:17.444838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3913149-9464-48ad-85c3-5f3b1d437b37 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:17.445044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19065ba6-bc68-41a9-b6ff-3fdc1435a262 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:17.445551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3913149-9464-48ad-85c3-5f3b1d437b37 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:17.446653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19065ba6-bc68-41a9-b6ff-3fdc1435a262 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:17.447209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3913149-9464-48ad-85c3-5f3b1d437b37 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:17.452028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 513/1015] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Aug 31 11:31:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:17.457691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19065ba6-bc68-41a9-b6ff-3fdc1435a262 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:17.479253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 503/1016] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:31:17 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:17.578994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19065ba6-bc68-41a9-b6ff-3fdc1435a262 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:17.617506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ea246-970c-499b-8b0f-03ad573c7b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:17.618350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ea246-970c-499b-8b0f-03ad573c7b81 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:17.619268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ea246-970c-499b-8b0f-03ad573c7b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:17.619821 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ea246-970c-499b-8b0f-03ad573c7b81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:17.620514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ea246-970c-499b-8b0f-03ad573c7b81 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:17.621877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ea246-970c-499b-8b0f-03ad573c7b81 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:17.643224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 514/1017] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:31:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:31:18.143504 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7833e7e-3c64-45ab-b4a9-ba80f3135ee8 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:18.144035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7833e7e-3c64-45ab-b4a9-ba80f3135ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:18.144035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7833e7e-3c64-45ab-b4a9-ba80f3135ee8 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:18.145574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 504/1018] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:18.151111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928349a-b612-4e86-b1f8-4216975255e5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:18.151401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928349a-b612-4e86-b1f8-4216975255e5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:18.151675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928349a-b612-4e86-b1f8-4216975255e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:18.170507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0928349a-b612-4e86-b1f8-4216975255e5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:18.171698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0928349a-b612-4e86-b1f8-4216975255e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:18.187875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:18.250459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 515/1019] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:18 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:18.514860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7833e7e-3c64-45ab-b4a9-ba80f3135ee8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:18.529887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:18.619447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7833e7e-3c64-45ab-b4a9-ba80f3135ee8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:18.655204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d065d04-2fb9-41ac-bde3-f90a4a331a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:18.655812 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d065d04-2fb9-41ac-bde3-f90a4a331a4b None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:18.656090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d065d04-2fb9-41ac-bde3-f90a4a331a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:18.656359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d065d04-2fb9-41ac-bde3-f90a4a331a4b None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:18.657031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d065d04-2fb9-41ac-bde3-f90a4a331a4b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:18.658385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d065d04-2fb9-41ac-bde3-f90a4a331a4b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:18.673242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 505/1020] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:31:18 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:19.471664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3661bc-52ed-49d2-8361-8dd68645863b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:19.472237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3661bc-52ed-49d2-8361-8dd68645863b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:19.472237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3661bc-52ed-49d2-8361-8dd68645863b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:19.473598 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 516/1021] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:19.478641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f4d12-2dc4-4533-b149-2bea907ed5a7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:19.479025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f4d12-2dc4-4533-b149-2bea907ed5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:19.479202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f4d12-2dc4-4533-b149-2bea907ed5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:19.497575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ea9f4d12-2dc4-4533-b149-2bea907ed5a7 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:19.498749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ea9f4d12-2dc4-4533-b149-2bea907ed5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:19.565870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:19.576649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 506/1022] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:19.678500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e87009-2be6-4115-8bab-78a92912c8d8 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:19.678843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e87009-2be6-4115-8bab-78a92912c8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:19.679081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e87009-2be6-4115-8bab-78a92912c8d8 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:19.680682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 517/1023] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:19.687235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5bd9a-7afd-4ffb-94ea-9ac40b04cb82 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:19.687599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5bd9a-7afd-4ffb-94ea-9ac40b04cb82 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:19.687949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5bd9a-7afd-4ffb-94ea-9ac40b04cb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:19.706412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2cb5bd9a-7afd-4ffb-94ea-9ac40b04cb82 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:19.707601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2cb5bd9a-7afd-4ffb-94ea-9ac40b04cb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:19.778803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 507/1024] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:19.840738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e87009-2be6-4115-8bab-78a92912c8d8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:19.944649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e87009-2be6-4115-8bab-78a92912c8d8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:19.979733 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ecf56-b08d-4a28-8722-5783982760b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:19.980351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ecf56-b08d-4a28-8722-5783982760b1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:19.980662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ecf56-b08d-4a28-8722-5783982760b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:19.980934 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ecf56-b08d-4a28-8722-5783982760b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:19.982034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ecf56-b08d-4a28-8722-5783982760b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:19.983446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ecf56-b08d-4a28-8722-5783982760b1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:19.988532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 518/1025] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:19.990694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5bd9a-7afd-4ffb-94ea-9ac40b04cb82 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:20.044201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ecf56-b08d-4a28-8722-5783982760b1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:20.060283 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:20.104120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5bd9a-7afd-4ffb-94ea-9ac40b04cb82 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:20.132789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:20.141762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484dc32-029f-42ef-945c-4352a05e30be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:20.142370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484dc32-029f-42ef-945c-4352a05e30be None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:20.142654 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484dc32-029f-42ef-945c-4352a05e30be None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:20.142935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484dc32-029f-42ef-945c-4352a05e30be None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:20.143594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484dc32-029f-42ef-945c-4352a05e30be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:20.145073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484dc32-029f-42ef-945c-4352a05e30be None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:20.152302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ecf56-b08d-4a28-8722-5783982760b1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:20.172346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 508/1026] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:31:19 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:20.195961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28ec7f-f929-430d-883c-13ad5a687aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:20.197570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28ec7f-f929-430d-883c-13ad5a687aea None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:20.198031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28ec7f-f929-430d-883c-13ad5a687aea None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:20.198451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28ec7f-f929-430d-883c-13ad5a687aea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:20.212782 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28ec7f-f929-430d-883c-13ad5a687aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0a28ec7f-f929-430d-883c-13ad5a687aea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:20.218307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:20.219567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:20.220632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:20.224564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 519/1027] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Aug 31 11:31:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:20.228928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484dc32-029f-42ef-945c-4352a05e30be None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:20.344192 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484dc32-029f-42ef-945c-4352a05e30be None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:20.382603 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a157e-b95d-43f2-970a-483e547b303d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:20.383217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a157e-b95d-43f2-970a-483e547b303d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:20.383500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a157e-b95d-43f2-970a-483e547b303d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:20.383775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a157e-b95d-43f2-970a-483e547b303d None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:20.384473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a157e-b95d-43f2-970a-483e547b303d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:20.384817 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a157e-b95d-43f2-970a-483e547b303d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:20.386196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a157e-b95d-43f2-970a-483e547b303d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:20.395099 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 509/1028] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Aug 31 11:31:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:20.401182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28ec7f-f929-430d-883c-13ad5a687aea None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:20.482245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:20.510070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28ec7f-f929-430d-883c-13ad5a687aea None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:20.544520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43852579-a8f4-419b-b114-78b011ee56b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:20.545198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43852579-a8f4-419b-b114-78b011ee56b4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:20.545484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43852579-a8f4-419b-b114-78b011ee56b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:20.545820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43852579-a8f4-419b-b114-78b011ee56b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:20.556777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43852579-a8f4-419b-b114-78b011ee56b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:20.558213 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43852579-a8f4-419b-b114-78b011ee56b4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-43852579-a8f4-419b-b114-78b011ee56b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:20.573538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:20.574524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:20.575439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:20.575439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:20.575439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:20.575439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 520/1029] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:20.580442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a157e-b95d-43f2-970a-483e547b303d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:20.602828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdfe1d-b55f-44cc-b89a-98f80dce3d36 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:20.603391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdfe1d-b55f-44cc-b89a-98f80dce3d36 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:20.603667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdfe1d-b55f-44cc-b89a-98f80dce3d36 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:20.605036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 521/1030] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:20.610067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04e39f-33f1-4b62-b1d9-75427f1d9208 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:20.610346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04e39f-33f1-4b62-b1d9-75427f1d9208 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:20.610620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04e39f-33f1-4b62-b1d9-75427f1d9208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:20.628635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1d04e39f-33f1-4b62-b1d9-75427f1d9208 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:20.629821 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1d04e39f-33f1-4b62-b1d9-75427f1d9208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:20.637607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:20.699311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a157e-b95d-43f2-970a-483e547b303d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:20.707164 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 522/1031] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:20.735270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a140c3-fab7-4d04-b45c-f1fd618004ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:20.735877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a140c3-fab7-4d04-b45c-f1fd618004ae None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:20.736161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a140c3-fab7-4d04-b45c-f1fd618004ae None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:20.736436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a140c3-fab7-4d04-b45c-f1fd618004ae None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:20.737113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a140c3-fab7-4d04-b45c-f1fd618004ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:20.737419 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a140c3-fab7-4d04-b45c-f1fd618004ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:20.738783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a140c3-fab7-4d04-b45c-f1fd618004ae None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:20.757529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 510/1032] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:20.758684 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04e39f-33f1-4b62-b1d9-75427f1d9208 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:20.820316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:20.865201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04e39f-33f1-4b62-b1d9-75427f1d9208 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:20.899171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33489a4d-0436-410e-ba1a-b11c4a58d336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:20.900394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33489a4d-0436-410e-ba1a-b11c4a58d336 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:20.900701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33489a4d-0436-410e-ba1a-b11c4a58d336 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:20.900968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33489a4d-0436-410e-ba1a-b11c4a58d336 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:20.907972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33489a4d-0436-410e-ba1a-b11c4a58d336 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:20.909426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33489a4d-0436-410e-ba1a-b11c4a58d336 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-33489a4d-0436-410e-ba1a-b11c4a58d336 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:31:20.921394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:20.922340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:20.928505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:20.928505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:20.928505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:31:20.928505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:31:20.928505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:31:20.928505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:31:20.928505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:31:20.928505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:31:20.928505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:31:20.928505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:20.928505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 523/1033] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Aug 31 11:31:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:20.930464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a140c3-fab7-4d04-b45c-f1fd618004ae None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:20.979236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33489a4d-0436-410e-ba1a-b11c4a58d336 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:21.038959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a140c3-fab7-4d04-b45c-f1fd618004ae None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:21.049226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:21.081916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770406b9-8cfe-4b45-a8f5-56a5eeba3e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:21.082896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770406b9-8cfe-4b45-a8f5-56a5eeba3e99 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:21.083400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770406b9-8cfe-4b45-a8f5-56a5eeba3e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:21.084059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770406b9-8cfe-4b45-a8f5-56a5eeba3e99 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:21.085014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770406b9-8cfe-4b45-a8f5-56a5eeba3e99 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:21.085508 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770406b9-8cfe-4b45-a8f5-56a5eeba3e99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:21.087408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770406b9-8cfe-4b45-a8f5-56a5eeba3e99 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:21.092878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33489a4d-0436-410e-ba1a-b11c4a58d336 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:21.113937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 511/1034] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Aug 31 11:31:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:21.114132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:21.114998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770406b9-8cfe-4b45-a8f5-56a5eeba3e99 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:21.132721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c4e2-b9dc-41dd-8dcc-e3f98ec78af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:21.133334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c4e2-b9dc-41dd-8dcc-e3f98ec78af1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:21.133623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c4e2-b9dc-41dd-8dcc-e3f98ec78af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:21.134698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c4e2-b9dc-41dd-8dcc-e3f98ec78af1 None admin] PATH_INFO: `/v3/services/image` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:21.135512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235c4e2-b9dc-41dd-8dcc-e3f98ec78af1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:21.136921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235c4e2-b9dc-41dd-8dcc-e3f98ec78af1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c235c4e2-b9dc-41dd-8dcc-e3f98ec78af1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:21.150019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:21.151019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:21.151864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 524/1035] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Aug 31 11:31:20 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:21.157918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c4e2-b9dc-41dd-8dcc-e3f98ec78af1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:21.233760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770406b9-8cfe-4b45-a8f5-56a5eeba3e99 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:21.268287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33032015-bcbc-4239-baec-79e4353e7eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:21.268922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032015-bcbc-4239-baec-79e4353e7eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:21.269108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c4e2-b9dc-41dd-8dcc-e3f98ec78af1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:21.269220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032015-bcbc-4239-baec-79e4353e7eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:21.269459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032015-bcbc-4239-baec-79e4353e7eb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:21.304250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:21.311907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c38ad-468d-437e-8bd8-1edfda85ea79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:21.312816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c38ad-468d-437e-8bd8-1edfda85ea79 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:21.313262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c38ad-468d-437e-8bd8-1edfda85ea79 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:21.313682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c38ad-468d-437e-8bd8-1edfda85ea79 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:21.314911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c38ad-468d-437e-8bd8-1edfda85ea79 None admin] RBAC: Adding query filter params (name=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:21.315367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c38ad-468d-437e-8bd8-1edfda85ea79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:21.317983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c38ad-468d-437e-8bd8-1edfda85ea79 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:21.329184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 525/1036] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:21.330382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c38ad-468d-437e-8bd8-1edfda85ea79 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:21.346496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33032015-bcbc-4239-baec-79e4353e7eb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:21.347964 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33032015-bcbc-4239-baec-79e4353e7eb3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:21.378610 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:21.440319 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c38ad-468d-437e-8bd8-1edfda85ea79 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:21.459381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 512/1037] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:21 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:31:21.460676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33032015-bcbc-4239-baec-79e4353e7eb3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:21.482900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaddf7-f84f-4c9d-9a25-4192bcd768ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:21.483495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaddf7-f84f-4c9d-9a25-4192bcd768ee None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:21.483773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaddf7-f84f-4c9d-9a25-4192bcd768ee None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:21.484040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaddf7-f84f-4c9d-9a25-4192bcd768ee None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:21.484685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaddf7-f84f-4c9d-9a25-4192bcd768ee None admin] RBAC: Adding query filter params (type=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:21.484983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaddf7-f84f-4c9d-9a25-4192bcd768ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:21.486337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaddf7-f84f-4c9d-9a25-4192bcd768ee None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:21.496745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 526/1038] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:21.497884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaddf7-f84f-4c9d-9a25-4192bcd768ee None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:21.529889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:21.570562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33032015-bcbc-4239-baec-79e4353e7eb3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:21.605388 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaddf7-f84f-4c9d-9a25-4192bcd768ee None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:21.608289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dc9a9-d4ee-4548-aaba-d6c0191ff240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:21.608954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dc9a9-d4ee-4548-aaba-d6c0191ff240 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:21.609234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dc9a9-d4ee-4548-aaba-d6c0191ff240 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:21.609508 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dc9a9-d4ee-4548-aaba-d6c0191ff240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:21.614659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:21.622470 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dc9a9-d4ee-4548-aaba-d6c0191ff240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:21.624146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dc9a9-d4ee-4548-aaba-d6c0191ff240 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-938dc9a9-d4ee-4548-aaba-d6c0191ff240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:21.639700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:21.640709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:21.641604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:21.641604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:21.641604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:21.641604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 513/1039] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:21.646284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dc9a9-d4ee-4548-aaba-d6c0191ff240 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:21.647500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba80c84-3424-47d5-8f8c-7fcd92ca9348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:21.648114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba80c84-3424-47d5-8f8c-7fcd92ca9348 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:21.648397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba80c84-3424-47d5-8f8c-7fcd92ca9348 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:21.648720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba80c84-3424-47d5-8f8c-7fcd92ca9348 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:21.649948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba80c84-3424-47d5-8f8c-7fcd92ca9348 None admin] RBAC: Adding query filter params (interface=public, service_id=4a961970362342b7b6aab9aa086dcf63, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:21.650461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba80c84-3424-47d5-8f8c-7fcd92ca9348 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:21.651936 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba80c84-3424-47d5-8f8c-7fcd92ca9348 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:21.657165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 527/1040] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Aug 31 11:31:21 2022] GET /identity/v3/endpoints?service_id=4a961970362342b7b6aab9aa086dcf63&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:21.662576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba80c84-3424-47d5-8f8c-7fcd92ca9348 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:21.760884 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dc9a9-d4ee-4548-aaba-d6c0191ff240 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:21.767556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba80c84-3424-47d5-8f8c-7fcd92ca9348 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:21.778330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:21.798901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fec7e4-214e-487b-8c29-757876a59575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:21.799517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fec7e4-214e-487b-8c29-757876a59575 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:21.799806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fec7e4-214e-487b-8c29-757876a59575 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:21.800098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fec7e4-214e-487b-8c29-757876a59575 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:21.800738 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fec7e4-214e-487b-8c29-757876a59575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:21.801040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fec7e4-214e-487b-8c29-757876a59575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:21.802479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fec7e4-214e-487b-8c29-757876a59575 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:21.807115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22421298-882b-44b8-8c6d-e91c2d4ccbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:21.807724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22421298-882b-44b8-8c6d-e91c2d4ccbdb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:21.808007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22421298-882b-44b8-8c6d-e91c2d4ccbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:21.808286 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22421298-882b-44b8-8c6d-e91c2d4ccbdb None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:21.808923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22421298-882b-44b8-8c6d-e91c2d4ccbdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:21.810279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22421298-882b-44b8-8c6d-e91c2d4ccbdb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:21.820254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 514/1041] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:21.820437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 528/1042] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Aug 31 11:31:21 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:31:21.825651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fec7e4-214e-487b-8c29-757876a59575 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:21.936765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fec7e4-214e-487b-8c29-757876a59575 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:21.974249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f734c-36e3-458b-af1c-4bd950e4e318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:21.974879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f734c-36e3-458b-af1c-4bd950e4e318 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:21.975154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f734c-36e3-458b-af1c-4bd950e4e318 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:21.975422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f734c-36e3-458b-af1c-4bd950e4e318 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:21.982999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f734c-36e3-458b-af1c-4bd950e4e318 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:21.984447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f734c-36e3-458b-af1c-4bd950e4e318 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c57f734c-36e3-458b-af1c-4bd950e4e318 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:31:21.995815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:21.996928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:21.997848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:31:21.997848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:31:21.997848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:31:21.997848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:31:21.997848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:31:21.997848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:31:21.997848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:21.997848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:21.997848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 515/1043] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:31:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:22.002741 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22421298-882b-44b8-8c6d-e91c2d4ccbdb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:22.045946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:22.113570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22421298-882b-44b8-8c6d-e91c2d4ccbdb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:22.148883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadb01b-7371-476b-af94-0f08ae0511ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:22.150097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadb01b-7371-476b-af94-0f08ae0511ba None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:22.150460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadb01b-7371-476b-af94-0f08ae0511ba None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:22.150741 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadb01b-7371-476b-af94-0f08ae0511ba None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:22.151427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadb01b-7371-476b-af94-0f08ae0511ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:22.151732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadb01b-7371-476b-af94-0f08ae0511ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:22.153189 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadb01b-7371-476b-af94-0f08ae0511ba None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:22.166061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 529/1044] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:31:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:22.171930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f734c-36e3-458b-af1c-4bd950e4e318 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:22.180011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e2ea5-b116-403a-aaaa-0d4b61138d92 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:22.180295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e2ea5-b116-403a-aaaa-0d4b61138d92 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:22.180596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e2ea5-b116-403a-aaaa-0d4b61138d92 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:22.182109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 530/1045] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:31:22.187262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c6922-d768-4b6c-bd7d-89944e8c404c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:22.187547 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c6922-d768-4b6c-bd7d-89944e8c404c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:22.187823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c6922-d768-4b6c-bd7d-89944e8c404c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:22.206444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d50c6922-d768-4b6c-bd7d-89944e8c404c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:22.207619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d50c6922-d768-4b6c-bd7d-89944e8c404c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:22.214579 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:22.284546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 531/1046] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:22.287033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f734c-36e3-458b-af1c-4bd950e4e318 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:22.322508 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574ce79-e9c3-48a5-bbaf-34b928a1a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:22.323108 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574ce79-e9c3-48a5-bbaf-34b928a1a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:22.323386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574ce79-e9c3-48a5-bbaf-34b928a1a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:22.323645 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574ce79-e9c3-48a5-bbaf-34b928a1a9a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:22.324361 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574ce79-e9c3-48a5-bbaf-34b928a1a9a0 None admin] RBAC: Adding query filter params (role.id=455a7f6a98b44607998cf25d8dec97cc, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=f4048316ea784c389eb249b4dd915e0e) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:22.324695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574ce79-e9c3-48a5-bbaf-34b928a1a9a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:22.326117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574ce79-e9c3-48a5-bbaf-34b928a1a9a0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:22.340779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 516/1047] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:31:22 2022] GET /identity/v3/role_assignments?user.id=f4048316ea784c389eb249b4dd915e0e&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=455a7f6a98b44607998cf25d8dec97cc => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:22.553569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c6922-d768-4b6c-bd7d-89944e8c404c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:22.582857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:22.657434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c6922-d768-4b6c-bd7d-89944e8c404c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:22.701038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30889a-228e-42ed-af1d-7001e3704c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:22.701664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30889a-228e-42ed-af1d-7001e3704c67 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:22.701942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30889a-228e-42ed-af1d-7001e3704c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:22.702213 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30889a-228e-42ed-af1d-7001e3704c67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:22.712834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30889a-228e-42ed-af1d-7001e3704c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2e30889a-228e-42ed-af1d-7001e3704c67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:22.717668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:22.718574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:22.719436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:22.723672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 532/1048] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:31:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:22.724685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574ce79-e9c3-48a5-bbaf-34b928a1a9a0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:22.828163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:22.831402 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574ce79-e9c3-48a5-bbaf-34b928a1a9a0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:22.867289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ef4c6-9fdb-422b-b425-e42d4cd9328e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:22.867814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ef4c6-9fdb-422b-b425-e42d4cd9328e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:22.868100 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ef4c6-9fdb-422b-b425-e42d4cd9328e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:22.868374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ef4c6-9fdb-422b-b425-e42d4cd9328e None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:22.869066 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ef4c6-9fdb-422b-b425-e42d4cd9328e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:22.869370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ef4c6-9fdb-422b-b425-e42d4cd9328e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:22.870736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ef4c6-9fdb-422b-b425-e42d4cd9328e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:22.879573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 517/1049] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:31:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:22.885681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30889a-228e-42ed-af1d-7001e3704c67 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:22.975307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:22.991724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30889a-228e-42ed-af1d-7001e3704c67 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:23.027195 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274743e-775c-4014-b9ac-bc9a48648ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:23.027812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274743e-775c-4014-b9ac-bc9a48648ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:23.028090 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274743e-775c-4014-b9ac-bc9a48648ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:23.028365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274743e-775c-4014-b9ac-bc9a48648ef5 None admin] PATH_INFO: `/v3/users/a61816963ad447f3bff39ea6f7090d40` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:23.041708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274743e-775c-4014-b9ac-bc9a48648ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=a61816963ad447f3bff39ea6f7090d40)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:23.043128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274743e-775c-4014-b9ac-bc9a48648ef5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:23.044962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 533/1050] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:22 2022] GET /identity/v3/users/a61816963ad447f3bff39ea6f7090d40 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:23.050564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ef4c6-9fdb-422b-b425-e42d4cd9328e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:23.148886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ef4c6-9fdb-422b-b425-e42d4cd9328e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:23.184284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a92f51-2e79-4dad-ae65-608718cc0176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:23.184931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a92f51-2e79-4dad-ae65-608718cc0176 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:23.185215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a92f51-2e79-4dad-ae65-608718cc0176 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:23.185492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a92f51-2e79-4dad-ae65-608718cc0176 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:23.215976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:23.255423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a92f51-2e79-4dad-ae65-608718cc0176 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:23.256910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a92f51-2e79-4dad-ae65-608718cc0176 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:23.357784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 518/1051] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:23 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:31:23.363375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274743e-775c-4014-b9ac-bc9a48648ef5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:23.465656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274743e-775c-4014-b9ac-bc9a48648ef5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:23.499545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa3d3c-ca8a-4964-937b-e2ef0f3fd317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:23.500170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa3d3c-ca8a-4964-937b-e2ef0f3fd317 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:23.500449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa3d3c-ca8a-4964-937b-e2ef0f3fd317 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:23.500746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa3d3c-ca8a-4964-937b-e2ef0f3fd317 None admin] PATH_INFO: `/v3/projects/5ca780a28b784f98a577c8e249cd0f59` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:23.501842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa3d3c-ca8a-4964-937b-e2ef0f3fd317 None admin] RBAC: Authorizing `identity:get_project(project_id=5ca780a28b784f98a577c8e249cd0f59)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:23.503254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa3d3c-ca8a-4964-937b-e2ef0f3fd317 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:23.510377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 534/1052] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:31:23 2022] GET /identity/v3/projects/5ca780a28b784f98a577c8e249cd0f59 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:23.512235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a92f51-2e79-4dad-ae65-608718cc0176 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:23.574555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:23.614307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a92f51-2e79-4dad-ae65-608718cc0176 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:23.648461 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51042a-5c90-4ab0-b2dd-453dede30bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:23.649085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51042a-5c90-4ab0-b2dd-453dede30bbd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:23.649365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51042a-5c90-4ab0-b2dd-453dede30bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:23.650388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51042a-5c90-4ab0-b2dd-453dede30bbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:23.651119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51042a-5c90-4ab0-b2dd-453dede30bbd None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=5ca780a28b784f98a577c8e249cd0f59, user.id=a61816963ad447f3bff39ea6f7090d40) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:23.651430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51042a-5c90-4ab0-b2dd-453dede30bbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:23.652888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51042a-5c90-4ab0-b2dd-453dede30bbd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:23.664581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 519/1053] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:31:23 2022] GET /identity/v3/role_assignments?user.id=a61816963ad447f3bff39ea6f7090d40&scope.project.id=5ca780a28b784f98a577c8e249cd0f59&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:23.745594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1f2d4-d37d-4d46-9f55-db90541bfa24 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:23.746121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1f2d4-d37d-4d46-9f55-db90541bfa24 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:23.746121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1f2d4-d37d-4d46-9f55-db90541bfa24 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:23.747421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 535/1054] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:23.753090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b17e8-ec85-4b28-9e11-b2f1b2dbe7cb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:23.753584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b17e8-ec85-4b28-9e11-b2f1b2dbe7cb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:23.753661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b17e8-ec85-4b28-9e11-b2f1b2dbe7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:23.773302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4e1b17e8-ec85-4b28-9e11-b2f1b2dbe7cb None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:23.775126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4e1b17e8-ec85-4b28-9e11-b2f1b2dbe7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:23.846757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 520/1055] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:24.116315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1f2d4-d37d-4d46-9f55-db90541bfa24 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:24.131364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:24.206158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa41f3d-e0fe-4f05-8658-662a9bd5209f None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:24.206481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa41f3d-e0fe-4f05-8658-662a9bd5209f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:24.206717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa41f3d-e0fe-4f05-8658-662a9bd5209f None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:24.208190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 521/1056] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:31:24.213774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57533f5-5a6e-4be5-a1f4-631864ec5488 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:24.214134 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57533f5-5a6e-4be5-a1f4-631864ec5488 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:24.214483 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57533f5-5a6e-4be5-a1f4-631864ec5488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:24.228835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1f2d4-d37d-4d46-9f55-db90541bfa24 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:24.233264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f57533f5-5a6e-4be5-a1f4-631864ec5488 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:24.234448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f57533f5-5a6e-4be5-a1f4-631864ec5488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:24.268424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800346b0-0dd7-4916-8eef-106931b6bdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:24.269081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800346b0-0dd7-4916-8eef-106931b6bdec None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:24.269362 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800346b0-0dd7-4916-8eef-106931b6bdec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:24.269633 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800346b0-0dd7-4916-8eef-106931b6bdec None admin] PATH_INFO: `/v3/services/image` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:24.270210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800346b0-0dd7-4916-8eef-106931b6bdec None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:24.271574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800346b0-0dd7-4916-8eef-106931b6bdec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-800346b0-0dd7-4916-8eef-106931b6bdec None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:24.281146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:24.282168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:24.283851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 536/1057] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Aug 31 11:31:24 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:24.288058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800346b0-0dd7-4916-8eef-106931b6bdec None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:24.294899 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:24.320572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 522/1058] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:24.366501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:24.390770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800346b0-0dd7-4916-8eef-106931b6bdec None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:24.424132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec29f4-a9c5-43f3-b1fe-29c12435abf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:24.424748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec29f4-a9c5-43f3-b1fe-29c12435abf6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:24.425026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec29f4-a9c5-43f3-b1fe-29c12435abf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:24.425293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec29f4-a9c5-43f3-b1fe-29c12435abf6 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:24.425910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec29f4-a9c5-43f3-b1fe-29c12435abf6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:24.426209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec29f4-a9c5-43f3-b1fe-29c12435abf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:24.427680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec29f4-a9c5-43f3-b1fe-29c12435abf6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:24.438055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 537/1059] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:24.439290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57533f5-5a6e-4be5-a1f4-631864ec5488 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:24.543642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57533f5-5a6e-4be5-a1f4-631864ec5488 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:24.578781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600a0d7-206b-4f6a-92ac-b9e52d65818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:24.579391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600a0d7-206b-4f6a-92ac-b9e52d65818d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:24.579688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600a0d7-206b-4f6a-92ac-b9e52d65818d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:24.580064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600a0d7-206b-4f6a-92ac-b9e52d65818d None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:24.580730 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600a0d7-206b-4f6a-92ac-b9e52d65818d None admin] RBAC: Adding query filter params (type=image) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:24.581039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600a0d7-206b-4f6a-92ac-b9e52d65818d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:24.582413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600a0d7-206b-4f6a-92ac-b9e52d65818d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:24.592415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 523/1060] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:24.592606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec29f4-a9c5-43f3-b1fe-29c12435abf6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:24.594097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600a0d7-206b-4f6a-92ac-b9e52d65818d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:24.647732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:24.704739 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600a0d7-206b-4f6a-92ac-b9e52d65818d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:24.709748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec29f4-a9c5-43f3-b1fe-29c12435abf6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:24.745795 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e6ebb-14d4-4e08-ac7e-59e6d8909b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:24.746432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4551f-5522-4d91-8931-f2dd94fec647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:24.746560 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e6ebb-14d4-4e08-ac7e-59e6d8909b3e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:24.746782 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e6ebb-14d4-4e08-ac7e-59e6d8909b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:24.746941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e6ebb-14d4-4e08-ac7e-59e6d8909b3e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:24.747032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4551f-5522-4d91-8931-f2dd94fec647 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:24.747253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4551f-5522-4d91-8931-f2dd94fec647 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:24.747724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4551f-5522-4d91-8931-f2dd94fec647 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:24.748401 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4551f-5522-4d91-8931-f2dd94fec647 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:24.749803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4551f-5522-4d91-8931-f2dd94fec647 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:24.754814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:24.758659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e6ebb-14d4-4e08-ac7e-59e6d8909b3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-637e6ebb-14d4-4e08-ac7e-59e6d8909b3e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:24.764711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:24.771237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 538/1061] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Aug 31 11:31:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:24.772159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e6ebb-14d4-4e08-ac7e-59e6d8909b3e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:24.789183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 524/1062] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:31:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:31:24.884261 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e6ebb-14d4-4e08-ac7e-59e6d8909b3e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:24.926923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e161374-0665-4eb2-b362-de8fe6f6fd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:24.927449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e161374-0665-4eb2-b362-de8fe6f6fd2a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:24.927725 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e161374-0665-4eb2-b362-de8fe6f6fd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:24.927994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e161374-0665-4eb2-b362-de8fe6f6fd2a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:24.928676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e161374-0665-4eb2-b362-de8fe6f6fd2a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:24.928973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e161374-0665-4eb2-b362-de8fe6f6fd2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:24.930330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e161374-0665-4eb2-b362-de8fe6f6fd2a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:24.939786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 539/1063] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Aug 31 11:31:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:24.945558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4551f-5522-4d91-8931-f2dd94fec647 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:25.051024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4551f-5522-4d91-8931-f2dd94fec647 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:25.058635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:25.088743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b81c84-7c7e-4724-b703-5569dd91f41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:25.089365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b81c84-7c7e-4724-b703-5569dd91f41b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:25.089641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b81c84-7c7e-4724-b703-5569dd91f41b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:25.089908 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b81c84-7c7e-4724-b703-5569dd91f41b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:25.100673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b81c84-7c7e-4724-b703-5569dd91f41b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:25.102125 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b81c84-7c7e-4724-b703-5569dd91f41b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b0b81c84-7c7e-4724-b703-5569dd91f41b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:25.117234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:25.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:25.120065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:25.120065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:25.120065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:25.120065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 525/1064] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:25.128174 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e161374-0665-4eb2-b362-de8fe6f6fd2a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:25.194361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:25.238673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e161374-0665-4eb2-b362-de8fe6f6fd2a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:25.276721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc8e02-d8f2-4ffa-bae9-3158969f020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:25.277335 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc8e02-d8f2-4ffa-bae9-3158969f020b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:25.277611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc8e02-d8f2-4ffa-bae9-3158969f020b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:25.277879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc8e02-d8f2-4ffa-bae9-3158969f020b None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:25.278514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc8e02-d8f2-4ffa-bae9-3158969f020b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:25.278814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc8e02-d8f2-4ffa-bae9-3158969f020b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:25.280158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc8e02-d8f2-4ffa-bae9-3158969f020b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:25.292102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 540/1065] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:25.297591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b81c84-7c7e-4724-b703-5569dd91f41b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:25.404059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b81c84-7c7e-4724-b703-5569dd91f41b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:25.438557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351872d-998e-4660-992e-cab086a39af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:25.439189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351872d-998e-4660-992e-cab086a39af0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:25.439471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351872d-998e-4660-992e-cab086a39af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:25.439749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351872d-998e-4660-992e-cab086a39af0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:25.447970 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351872d-998e-4660-992e-cab086a39af0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:25.449373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351872d-998e-4660-992e-cab086a39af0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-e351872d-998e-4660-992e-cab086a39af0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:31:25.460550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:25.461649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:25.464083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:25.464083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:25.464083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:31:25.464083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:31:25.464083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:31:25.464083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:31:25.464083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:31:25.464083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:31:25.464083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:31:25.464083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:25.464083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 526/1066] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Aug 31 11:31:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:25.468095 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc8e02-d8f2-4ffa-bae9-3158969f020b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:25.546283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62038155-8285-4cfb-b0ad-dbedc7540883 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:25.546569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62038155-8285-4cfb-b0ad-dbedc7540883 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:25.546849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62038155-8285-4cfb-b0ad-dbedc7540883 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:25.548321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 527/1067] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:31:25.552935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a235ef-2804-40b4-b30a-aab8bb0f4027 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:25.553220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a235ef-2804-40b4-b30a-aab8bb0f4027 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:25.553493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a235ef-2804-40b4-b30a-aab8bb0f4027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:25.572696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-28a235ef-2804-40b4-b30a-aab8bb0f4027 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:25.573881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-28a235ef-2804-40b4-b30a-aab8bb0f4027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:25.580016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc8e02-d8f2-4ffa-bae9-3158969f020b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:25.587385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:25.598647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:25.617889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af91b9-bb77-4923-ac89-5d8ec0dfcc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:25.619277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af91b9-bb77-4923-ac89-5d8ec0dfcc1c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:25.619735 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af91b9-bb77-4923-ac89-5d8ec0dfcc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:25.620021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af91b9-bb77-4923-ac89-5d8ec0dfcc1c None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:25.620789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af91b9-bb77-4923-ac89-5d8ec0dfcc1c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:25.621097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af91b9-bb77-4923-ac89-5d8ec0dfcc1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:25.622519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af91b9-bb77-4923-ac89-5d8ec0dfcc1c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:25.640572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 541/1068] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Aug 31 11:31:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:25.646301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af91b9-bb77-4923-ac89-5d8ec0dfcc1c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:25.650015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 528/1069] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:25.756484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af91b9-bb77-4923-ac89-5d8ec0dfcc1c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:25.796085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2acd9-e08b-4994-ac49-b4a1d166e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:25.796747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2acd9-e08b-4994-ac49-b4a1d166e36a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:25.797212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2acd9-e08b-4994-ac49-b4a1d166e36a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:25.797336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2acd9-e08b-4994-ac49-b4a1d166e36a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:25.809078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2acd9-e08b-4994-ac49-b4a1d166e36a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:25.810547 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2acd9-e08b-4994-ac49-b4a1d166e36a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ccc2acd9-e08b-4994-ac49-b4a1d166e36a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:25.826025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:25.827002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:25.827901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:25.827901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:25.827901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:25.831852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 542/1070] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:25.833289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a235ef-2804-40b4-b30a-aab8bb0f4027 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:25.914903 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2acd9-e08b-4994-ac49-b4a1d166e36a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:25.944006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a235ef-2804-40b4-b30a-aab8bb0f4027 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:25.947802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:25.980295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80cdc8-21b6-45c7-8986-77fb8ee1f1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:25.980931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80cdc8-21b6-45c7-8986-77fb8ee1f1fd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:25.981213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80cdc8-21b6-45c7-8986-77fb8ee1f1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:25.981486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80cdc8-21b6-45c7-8986-77fb8ee1f1fd None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:25.982122 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80cdc8-21b6-45c7-8986-77fb8ee1f1fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:25.982423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80cdc8-21b6-45c7-8986-77fb8ee1f1fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:25.983798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80cdc8-21b6-45c7-8986-77fb8ee1f1fd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:25.994716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 529/1071] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:26.000335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80cdc8-21b6-45c7-8986-77fb8ee1f1fd None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:26.011407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:26.039466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2acd9-e08b-4994-ac49-b4a1d166e36a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:26.077571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1858fd0-bc47-4266-b222-1fc8a446b3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:26.078421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1858fd0-bc47-4266-b222-1fc8a446b3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:26.078852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1858fd0-bc47-4266-b222-1fc8a446b3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:26.079348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1858fd0-bc47-4266-b222-1fc8a446b3e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:26.091327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1858fd0-bc47-4266-b222-1fc8a446b3e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e1858fd0-bc47-4266-b222-1fc8a446b3e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:26.096628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:26.097557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:26.098668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 543/1072] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:31:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:26.103610 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1858fd0-bc47-4266-b222-1fc8a446b3e8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:26.105705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80cdc8-21b6-45c7-8986-77fb8ee1f1fd None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:26.141241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fb6e9-e4a0-4c1c-98a9-410f9b6bda73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:26.141861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fb6e9-e4a0-4c1c-98a9-410f9b6bda73 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:26.142136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fb6e9-e4a0-4c1c-98a9-410f9b6bda73 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:26.142406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fb6e9-e4a0-4c1c-98a9-410f9b6bda73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:26.150128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fb6e9-e4a0-4c1c-98a9-410f9b6bda73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:26.151552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fb6e9-e4a0-4c1c-98a9-410f9b6bda73 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-bd5fb6e9-e4a0-4c1c-98a9-410f9b6bda73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:31:26.163229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:26.169179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:26.170352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:31:26.170352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:31:26.170352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:31:26.170352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:31:26.170352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:31:26.170352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:31:26.170352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:26.170352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:26.170352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 530/1073] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:31:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:26.170352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fb6e9-e4a0-4c1c-98a9-410f9b6bda73 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:26.203779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:26.217536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1858fd0-bc47-4266-b222-1fc8a446b3e8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:26.247180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:26.253170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc48b4-2619-4cfd-b816-f194bdb05523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:26.253775 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc48b4-2619-4cfd-b816-f194bdb05523 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:26.254048 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc48b4-2619-4cfd-b816-f194bdb05523 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:26.254316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc48b4-2619-4cfd-b816-f194bdb05523 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:26.254969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc48b4-2619-4cfd-b816-f194bdb05523 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:26.255268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc48b4-2619-4cfd-b816-f194bdb05523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:26.256643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc48b4-2619-4cfd-b816-f194bdb05523 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:26.265567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 544/1074] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:31:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:26.271118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc48b4-2619-4cfd-b816-f194bdb05523 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:26.291390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fb6e9-e4a0-4c1c-98a9-410f9b6bda73 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:26.326977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c8e00-1461-4cf4-b300-8dd6a607f406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:26.327575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c8e00-1461-4cf4-b300-8dd6a607f406 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:26.327854 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c8e00-1461-4cf4-b300-8dd6a607f406 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:26.328126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c8e00-1461-4cf4-b300-8dd6a607f406 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:26.328848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c8e00-1461-4cf4-b300-8dd6a607f406 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:26.329212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c8e00-1461-4cf4-b300-8dd6a607f406 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:26.330640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c8e00-1461-4cf4-b300-8dd6a607f406 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:26.343296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 531/1075] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:31:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:26.349123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c8e00-1461-4cf4-b300-8dd6a607f406 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:26.378235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc48b4-2619-4cfd-b816-f194bdb05523 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:26.415187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a53b6c-c45d-4ad5-90d2-6fd7f387e7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:26.415725 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a53b6c-c45d-4ad5-90d2-6fd7f387e7de None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:26.416005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a53b6c-c45d-4ad5-90d2-6fd7f387e7de None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:26.416274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a53b6c-c45d-4ad5-90d2-6fd7f387e7de None admin] PATH_INFO: `/v3/users/a61816963ad447f3bff39ea6f7090d40` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:26.417430 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a53b6c-c45d-4ad5-90d2-6fd7f387e7de None admin] RBAC: Authorizing `identity:get_user(user_id=a61816963ad447f3bff39ea6f7090d40)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:26.418858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a53b6c-c45d-4ad5-90d2-6fd7f387e7de None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:26.420633 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 545/1076] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:26 2022] GET /identity/v3/users/a61816963ad447f3bff39ea6f7090d40 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:26.426064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a53b6c-c45d-4ad5-90d2-6fd7f387e7de None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:26.446014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:26.459035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c8e00-1461-4cf4-b300-8dd6a607f406 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:26.487318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:26.496900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad3ffd-b22c-4610-96ee-52a351037889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:26.497553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad3ffd-b22c-4610-96ee-52a351037889 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:26.497856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad3ffd-b22c-4610-96ee-52a351037889 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:26.498144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad3ffd-b22c-4610-96ee-52a351037889 None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/f4048316ea784c389eb249b4dd915e0e/roles/455a7f6a98b44607998cf25d8dec97cc` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:26.500310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad3ffd-b22c-4610-96ee-52a351037889 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=f4048316ea784c389eb249b4dd915e0e, role_id=455a7f6a98b44607998cf25d8dec97cc)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:26.501769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad3ffd-b22c-4610-96ee-52a351037889 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:26.520674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 532/1077] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:31:26 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/f4048316ea784c389eb249b4dd915e0e/roles/455a7f6a98b44607998cf25d8dec97cc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:31:26.542394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a53b6c-c45d-4ad5-90d2-6fd7f387e7de None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:26.581020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b4766-eeb0-4294-acd7-4e0a346df908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:26.581691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b4766-eeb0-4294-acd7-4e0a346df908 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:26.581951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b4766-eeb0-4294-acd7-4e0a346df908 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:26.582224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b4766-eeb0-4294-acd7-4e0a346df908 None admin] PATH_INFO: `/v3/projects/5ca780a28b784f98a577c8e249cd0f59` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:26.583397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b4766-eeb0-4294-acd7-4e0a346df908 None admin] RBAC: Authorizing `identity:get_project(project_id=5ca780a28b784f98a577c8e249cd0f59)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:26.584865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b4766-eeb0-4294-acd7-4e0a346df908 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:26.586772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 546/1078] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:31:26 2022] GET /identity/v3/projects/5ca780a28b784f98a577c8e249cd0f59 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:26.592721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad3ffd-b22c-4610-96ee-52a351037889 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:26.712920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad3ffd-b22c-4610-96ee-52a351037889 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:26.748364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ac549-5f2d-4910-b367-1804c5d618ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:26.749050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ac549-5f2d-4910-b367-1804c5d618ee None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:26.749331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ac549-5f2d-4910-b367-1804c5d618ee None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:26.749604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ac549-5f2d-4910-b367-1804c5d618ee None admin] PATH_INFO: `/v3/projects/5ca780a28b784f98a577c8e249cd0f59/users/a61816963ad447f3bff39ea6f7090d40/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:26.751477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ac549-5f2d-4910-b367-1804c5d618ee None admin] RBAC: Authorizing `identity:create_grant(project_id=5ca780a28b784f98a577c8e249cd0f59, user_id=a61816963ad447f3bff39ea6f7090d40, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:26.752952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ac549-5f2d-4910-b367-1804c5d618ee None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:26.769436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 533/1079] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:31:26 2022] PUT /identity/v3/projects/5ca780a28b784f98a577c8e249cd0f59/users/a61816963ad447f3bff39ea6f7090d40/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:31:26.787862 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6619ef-9c45-4753-a2ac-eb7805890d8b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:26.788380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6619ef-9c45-4753-a2ac-eb7805890d8b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:26.788380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6619ef-9c45-4753-a2ac-eb7805890d8b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:26.789873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 547/1080] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:26.795823 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce592b-0662-4461-a1bd-4b88901c1e8b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:26.796129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce592b-0662-4461-a1bd-4b88901c1e8b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:26.796380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce592b-0662-4461-a1bd-4b88901c1e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:26.817426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-81ce592b-0662-4461-a1bd-4b88901c1e8b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:26.818647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-81ce592b-0662-4461-a1bd-4b88901c1e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:26.868803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:26.896397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 534/1081] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:27.169829 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6619ef-9c45-4753-a2ac-eb7805890d8b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:27.260255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:27.277667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6619ef-9c45-4753-a2ac-eb7805890d8b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:27.310758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf762d9-f770-4d42-a33b-33ad3ce44416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:27.311358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf762d9-f770-4d42-a33b-33ad3ce44416 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:27.311637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf762d9-f770-4d42-a33b-33ad3ce44416 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:27.311905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf762d9-f770-4d42-a33b-33ad3ce44416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:27.378756 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf762d9-f770-4d42-a33b-33ad3ce44416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:27.380197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf762d9-f770-4d42-a33b-33ad3ce44416 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:27.481265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:27.504637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 548/1082] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:31:27.504905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce592b-0662-4461-a1bd-4b88901c1e8b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:27.608583 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce592b-0662-4461-a1bd-4b88901c1e8b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:27.645361 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f5781-422a-4d5f-9384-ad79b37dd09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:27.646006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f5781-422a-4d5f-9384-ad79b37dd09f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:27.646291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f5781-422a-4d5f-9384-ad79b37dd09f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:27.646567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f5781-422a-4d5f-9384-ad79b37dd09f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:27.657190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f5781-422a-4d5f-9384-ad79b37dd09f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:27.658653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f5781-422a-4d5f-9384-ad79b37dd09f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-060f5781-422a-4d5f-9384-ad79b37dd09f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:27.673452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:27.674442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:27.675366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:27.675366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:27.675366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:27.681203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 535/1083] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:27.683086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf762d9-f770-4d42-a33b-33ad3ce44416 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:27.787854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf762d9-f770-4d42-a33b-33ad3ce44416 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:27.824588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d8dd6-9419-4328-bdb3-bdc8e4cfe911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:27.825226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d8dd6-9419-4328-bdb3-bdc8e4cfe911 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:27.825457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d8dd6-9419-4328-bdb3-bdc8e4cfe911 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:27.825725 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d8dd6-9419-4328-bdb3-bdc8e4cfe911 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:27.826361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d8dd6-9419-4328-bdb3-bdc8e4cfe911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:27.826666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d8dd6-9419-4328-bdb3-bdc8e4cfe911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:27.828022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d8dd6-9419-4328-bdb3-bdc8e4cfe911 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:27.848763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 549/1084] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:28.458381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d7921-0133-401b-8b03-05ff08cb60d3 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:28.458915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d7921-0133-401b-8b03-05ff08cb60d3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:28.458915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d7921-0133-401b-8b03-05ff08cb60d3 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:28.460262 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 536/1085] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:28.466066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ca513-7949-4663-af37-db5df6e8e544 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:28.466425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ca513-7949-4663-af37-db5df6e8e544 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:28.466773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ca513-7949-4663-af37-db5df6e8e544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:28.485640 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-796ca513-7949-4663-af37-db5df6e8e544 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:28.486827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-796ca513-7949-4663-af37-db5df6e8e544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:28.528572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:28.553550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 550/1086] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:28.720449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c536b38-311f-4745-94ed-579e4a2f9008 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:28.720784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c536b38-311f-4745-94ed-579e4a2f9008 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:28.721128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c536b38-311f-4745-94ed-579e4a2f9008 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:28.722542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 537/1087] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:28.728253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16581d-3273-4250-ba02-f91e8efad13b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:28.728649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16581d-3273-4250-ba02-f91e8efad13b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:28.728995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16581d-3273-4250-ba02-f91e8efad13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:28.747856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1b16581d-3273-4250-ba02-f91e8efad13b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:28.749599 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1b16581d-3273-4250-ba02-f91e8efad13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:28.818871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 551/1088] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:28.820364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c536b38-311f-4745-94ed-579e4a2f9008 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:28.850556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:28.923014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c536b38-311f-4745-94ed-579e4a2f9008 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:28.956502 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd2111-4d3e-4a28-8aa2-4b05eeecdb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:28.957147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd2111-4d3e-4a28-8aa2-4b05eeecdb42 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:28.957430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd2111-4d3e-4a28-8aa2-4b05eeecdb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:28.957707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd2111-4d3e-4a28-8aa2-4b05eeecdb42 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:28.967672 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd2111-4d3e-4a28-8aa2-4b05eeecdb42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-12dd2111-4d3e-4a28-8aa2-4b05eeecdb42 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:28.972574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:28.973696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:28.974540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 538/1089] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Aug 31 11:31:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:28.980128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16581d-3273-4250-ba02-f91e8efad13b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.077667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16581d-3273-4250-ba02-f91e8efad13b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.084242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd2111-4d3e-4a28-8aa2-4b05eeecdb42 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.085306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:29.113550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee4076-4b37-47ab-961e-7727913a56ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.114152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee4076-4b37-47ab-961e-7727913a56ce None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.114436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee4076-4b37-47ab-961e-7727913a56ce None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.114706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee4076-4b37-47ab-961e-7727913a56ce None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.115366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee4076-4b37-47ab-961e-7727913a56ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:29.115672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee4076-4b37-47ab-961e-7727913a56ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.117106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee4076-4b37-47ab-961e-7727913a56ce None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.126044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 552/1090] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Aug 31 11:31:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:29.131806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee4076-4b37-47ab-961e-7727913a56ce None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.176389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:29.187236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd2111-4d3e-4a28-8aa2-4b05eeecdb42 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.221775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0698629-b92e-4c69-9373-0a936dc10d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.222391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0698629-b92e-4c69-9373-0a936dc10d95 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.222720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0698629-b92e-4c69-9373-0a936dc10d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.222939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0698629-b92e-4c69-9373-0a936dc10d95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.231605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee4076-4b37-47ab-961e-7727913a56ce None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.232821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0698629-b92e-4c69-9373-0a936dc10d95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b0698629-b92e-4c69-9373-0a936dc10d95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:29.237564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:29.238476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:29.239324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 539/1091] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:29.245130 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0698629-b92e-4c69-9373-0a936dc10d95 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.267979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b334e-2a48-4d7e-8e48-b301c8902faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.268628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b334e-2a48-4d7e-8e48-b301c8902faa None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.268905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b334e-2a48-4d7e-8e48-b301c8902faa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.269169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b334e-2a48-4d7e-8e48-b301c8902faa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.279504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b334e-2a48-4d7e-8e48-b301c8902faa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.280956 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b334e-2a48-4d7e-8e48-b301c8902faa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d94b334e-2a48-4d7e-8e48-b301c8902faa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:29.294680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:29.295658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:29.297169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:29.297169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:29.297169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:29.297169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 553/1092] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:29.301453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b334e-2a48-4d7e-8e48-b301c8902faa None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.324546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:29.345203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0698629-b92e-4c69-9373-0a936dc10d95 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.379329 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639abeae-00eb-4e5d-bb54-4db50c8020cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.379918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639abeae-00eb-4e5d-bb54-4db50c8020cd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.380197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639abeae-00eb-4e5d-bb54-4db50c8020cd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.380464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639abeae-00eb-4e5d-bb54-4db50c8020cd None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.381154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639abeae-00eb-4e5d-bb54-4db50c8020cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:29.381453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639abeae-00eb-4e5d-bb54-4db50c8020cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.382813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639abeae-00eb-4e5d-bb54-4db50c8020cd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.391193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 540/1093] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:29.396914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639abeae-00eb-4e5d-bb54-4db50c8020cd None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.403716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b334e-2a48-4d7e-8e48-b301c8902faa None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.411595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:29.438257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfa86c-b939-4430-8ca3-5a8ab815a0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.438845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfa86c-b939-4430-8ca3-5a8ab815a0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.439118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfa86c-b939-4430-8ca3-5a8ab815a0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.439388 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfa86c-b939-4430-8ca3-5a8ab815a0c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.440013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfa86c-b939-4430-8ca3-5a8ab815a0c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:29.440312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfa86c-b939-4430-8ca3-5a8ab815a0c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.441687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfa86c-b939-4430-8ca3-5a8ab815a0c0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.458316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 554/1094] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:29.459430 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfa86c-b939-4430-8ca3-5a8ab815a0c0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.497622 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639abeae-00eb-4e5d-bb54-4db50c8020cd None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.530459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68942722-73cc-4598-8ea2-177de6fb5237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.531087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68942722-73cc-4598-8ea2-177de6fb5237 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.531367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68942722-73cc-4598-8ea2-177de6fb5237 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.531645 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68942722-73cc-4598-8ea2-177de6fb5237 None admin] PATH_INFO: `/v3/users/a61816963ad447f3bff39ea6f7090d40` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.532789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68942722-73cc-4598-8ea2-177de6fb5237 None admin] RBAC: Authorizing `identity:get_user(user_id=a61816963ad447f3bff39ea6f7090d40)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.534212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68942722-73cc-4598-8ea2-177de6fb5237 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.535787 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 541/1095] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/users/a61816963ad447f3bff39ea6f7090d40 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:29.538901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:29.542328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68942722-73cc-4598-8ea2-177de6fb5237 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.558804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfa86c-b939-4430-8ca3-5a8ab815a0c0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.592785 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70e271-f345-4a97-8ec3-ca3c4427b3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.593404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70e271-f345-4a97-8ec3-ca3c4427b3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.593690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70e271-f345-4a97-8ec3-ca3c4427b3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.593959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70e271-f345-4a97-8ec3-ca3c4427b3b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.601099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70e271-f345-4a97-8ec3-ca3c4427b3b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.603143 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70e271-f345-4a97-8ec3-ca3c4427b3b6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6e70e271-f345-4a97-8ec3-ca3c4427b3b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:31:29.614507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:29.615570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:29.616545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:29.616545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:29.616545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:31:29.616545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:31:29.616545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:31:29.616545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:31:29.616545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:31:29.616545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:31:29.616545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 11:31:29.616545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:29.621024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 555/1096] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:29.621849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70e271-f345-4a97-8ec3-ca3c4427b3b6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.622747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:29.643029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68942722-73cc-4598-8ea2-177de6fb5237 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.677162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68e545-abcf-4377-b53e-d441b3880862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.677771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68e545-abcf-4377-b53e-d441b3880862 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.678051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68e545-abcf-4377-b53e-d441b3880862 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.678326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68e545-abcf-4377-b53e-d441b3880862 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.723321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70e271-f345-4a97-8ec3-ca3c4427b3b6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.745221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68e545-abcf-4377-b53e-d441b3880862 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.746707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68e545-abcf-4377-b53e-d441b3880862 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.756729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc7d7a-05bd-409d-9ab6-738d6c1ca8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.757271 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc7d7a-05bd-409d-9ab6-738d6c1ca8af None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.757548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc7d7a-05bd-409d-9ab6-738d6c1ca8af None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.757818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc7d7a-05bd-409d-9ab6-738d6c1ca8af None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.758534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc7d7a-05bd-409d-9ab6-738d6c1ca8af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:29.759469 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc7d7a-05bd-409d-9ab6-738d6c1ca8af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.760915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc7d7a-05bd-409d-9ab6-738d6c1ca8af None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.770809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:29.780281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 556/1097] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:29.785996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc7d7a-05bd-409d-9ab6-738d6c1ca8af None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.830680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:29.850417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 542/1098] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 11:31:29.856852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68e545-abcf-4377-b53e-d441b3880862 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.885886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc7d7a-05bd-409d-9ab6-738d6c1ca8af None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.919520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9d26a-3d09-47a9-ad53-4fa147752072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.920116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9d26a-3d09-47a9-ad53-4fa147752072 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.920395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9d26a-3d09-47a9-ad53-4fa147752072 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.920700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9d26a-3d09-47a9-ad53-4fa147752072 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.953813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68e545-abcf-4377-b53e-d441b3880862 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.983868 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:29.987832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9d26a-3d09-47a9-ad53-4fa147752072 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.987967 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9a57a-fdf9-468d-8aa1-fe93b5965a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.988688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9a57a-fdf9-468d-8aa1-fe93b5965a15 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.988974 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9a57a-fdf9-468d-8aa1-fe93b5965a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.989253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9a57a-fdf9-468d-8aa1-fe93b5965a15 None admin] PATH_INFO: `/v3/projects/5ca780a28b784f98a577c8e249cd0f59` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.989481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9d26a-3d09-47a9-ad53-4fa147752072 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.990459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9a57a-fdf9-468d-8aa1-fe93b5965a15 None admin] RBAC: Authorizing `identity:get_project(project_id=5ca780a28b784f98a577c8e249cd0f59)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.991888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9a57a-fdf9-468d-8aa1-fe93b5965a15 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.993705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 543/1099] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/projects/5ca780a28b784f98a577c8e249cd0f59 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:29.999535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9a57a-fdf9-468d-8aa1-fe93b5965a15 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:30.049243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:30.090560 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 557/1100] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 11:31:30.095131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9d26a-3d09-47a9-ad53-4fa147752072 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:30.100991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9a57a-fdf9-468d-8aa1-fe93b5965a15 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:30.136094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c339f05-ef33-4d5d-9f30-d34111f8d443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:30.136719 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c339f05-ef33-4d5d-9f30-d34111f8d443 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:30.137002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c339f05-ef33-4d5d-9f30-d34111f8d443 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:30.137293 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c339f05-ef33-4d5d-9f30-d34111f8d443 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:30.138004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c339f05-ef33-4d5d-9f30-d34111f8d443 None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=5ca780a28b784f98a577c8e249cd0f59, user.id=a61816963ad447f3bff39ea6f7090d40) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:30.138312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c339f05-ef33-4d5d-9f30-d34111f8d443 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:30.139752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c339f05-ef33-4d5d-9f30-d34111f8d443 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:30.146711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 544/1101] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/role_assignments?user.id=a61816963ad447f3bff39ea6f7090d40&scope.project.id=5ca780a28b784f98a577c8e249cd0f59&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:30.194661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:30.199568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9d26a-3d09-47a9-ad53-4fa147752072 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:30.239990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc95dd1-0914-4454-a426-2e544929c237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:30.240661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95dd1-0914-4454-a426-2e544929c237 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:30.240939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95dd1-0914-4454-a426-2e544929c237 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:30.241208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95dd1-0914-4454-a426-2e544929c237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:30.252019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc95dd1-0914-4454-a426-2e544929c237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:30.253536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc95dd1-0914-4454-a426-2e544929c237 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2dc95dd1-0914-4454-a426-2e544929c237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:30.269386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:30.270409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:31:30.271329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:30.271329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:31:30.271329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:30.271329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 558/1102] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:31:30.277173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c339f05-ef33-4d5d-9f30-d34111f8d443 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:30.376326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c339f05-ef33-4d5d-9f30-d34111f8d443 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:30.389250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:30.410944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff615ae-3c37-4e26-bc4e-73e5c56856f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:30.411549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff615ae-3c37-4e26-bc4e-73e5c56856f7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:30.411825 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff615ae-3c37-4e26-bc4e-73e5c56856f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:30.412094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff615ae-3c37-4e26-bc4e-73e5c56856f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:30.412767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff615ae-3c37-4e26-bc4e-73e5c56856f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:30.413072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff615ae-3c37-4e26-bc4e-73e5c56856f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:30.414429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff615ae-3c37-4e26-bc4e-73e5c56856f7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:30.426229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 545/1103] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:30.431925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc95dd1-0914-4454-a426-2e544929c237 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:30.530267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc95dd1-0914-4454-a426-2e544929c237 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:30.562846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92815d44-6c5b-4a2a-bfec-9e4e71b63213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:30.563472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92815d44-6c5b-4a2a-bfec-9e4e71b63213 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:30.563754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92815d44-6c5b-4a2a-bfec-9e4e71b63213 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:30.564028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92815d44-6c5b-4a2a-bfec-9e4e71b63213 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:30.570839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92815d44-6c5b-4a2a-bfec-9e4e71b63213 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:30.572832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92815d44-6c5b-4a2a-bfec-9e4e71b63213 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-92815d44-6c5b-4a2a-bfec-9e4e71b63213 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:31:30.584761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:30.585792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:30.587426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:31:30.587426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:31:30.587426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:31:30.587426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:31:30.587426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:31:30.587426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:31:30.587426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:31:30.587426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:30.587426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 559/1104] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:31:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:30.592469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff615ae-3c37-4e26-bc4e-73e5c56856f7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:30.692646 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff615ae-3c37-4e26-bc4e-73e5c56856f7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:30.726850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cffff-1b4f-41a4-9062-d28f8a8edc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:30.727473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cffff-1b4f-41a4-9062-d28f8a8edc65 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:30.727768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cffff-1b4f-41a4-9062-d28f8a8edc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:30.728201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cffff-1b4f-41a4-9062-d28f8a8edc65 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:30.728712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cffff-1b4f-41a4-9062-d28f8a8edc65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:30.729022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cffff-1b4f-41a4-9062-d28f8a8edc65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:30.730443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cffff-1b4f-41a4-9062-d28f8a8edc65 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:30.741700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 546/1105] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:31:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:30.748108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92815d44-6c5b-4a2a-bfec-9e4e71b63213 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:30.761403 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:30.848928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92815d44-6c5b-4a2a-bfec-9e4e71b63213 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:30.880790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661fd32-0e47-427b-8a59-04f123be3dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:30.881373 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661fd32-0e47-427b-8a59-04f123be3dec None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:30.881648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661fd32-0e47-427b-8a59-04f123be3dec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:30.881916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661fd32-0e47-427b-8a59-04f123be3dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:30.882623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661fd32-0e47-427b-8a59-04f123be3dec None admin] RBAC: Adding query filter params (role.id=455a7f6a98b44607998cf25d8dec97cc, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=f4048316ea784c389eb249b4dd915e0e) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:30.882926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661fd32-0e47-427b-8a59-04f123be3dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:30.884953 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661fd32-0e47-427b-8a59-04f123be3dec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:30.890491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 560/1106] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:31:30 2022] GET /identity/v3/role_assignments?user.id=f4048316ea784c389eb249b4dd915e0e&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=455a7f6a98b44607998cf25d8dec97cc => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:32.086770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d84006-3120-4d84-83da-bb405c7b8473 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:32.087853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d84006-3120-4d84-83da-bb405c7b8473 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:32.087963 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d84006-3120-4d84-83da-bb405c7b8473 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:32.089305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 547/1107] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:32.094448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bca5d1-a5b9-4f1c-a316-7d0cc700943a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:32.094870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bca5d1-a5b9-4f1c-a316-7d0cc700943a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:32.094992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bca5d1-a5b9-4f1c-a316-7d0cc700943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:32.112669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d8bca5d1-a5b9-4f1c-a316-7d0cc700943a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:32.113826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d8bca5d1-a5b9-4f1c-a316-7d0cc700943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:32.178135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 561/1108] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:32.451267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d84006-3120-4d84-83da-bb405c7b8473 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:32.485570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:32.554218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d84006-3120-4d84-83da-bb405c7b8473 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:32.587591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607485d5-494f-45d2-964b-51f7b75c71a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:32.588198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607485d5-494f-45d2-964b-51f7b75c71a7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:32.588513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607485d5-494f-45d2-964b-51f7b75c71a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:32.588799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607485d5-494f-45d2-964b-51f7b75c71a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:32.589894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607485d5-494f-45d2-964b-51f7b75c71a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:32.591301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607485d5-494f-45d2-964b-51f7b75c71a7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:32.592861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 548/1109] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:32.598639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bca5d1-a5b9-4f1c-a316-7d0cc700943a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:32.601691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:32.694499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bca5d1-a5b9-4f1c-a316-7d0cc700943a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:32.728210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c133b-ff5f-4419-80c4-74f4d13427dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:32.728774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c133b-ff5f-4419-80c4-74f4d13427dc None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:32.729059 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c133b-ff5f-4419-80c4-74f4d13427dc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:32.729331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c133b-ff5f-4419-80c4-74f4d13427dc None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:32.729976 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c133b-ff5f-4419-80c4-74f4d13427dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:32.731386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c133b-ff5f-4419-80c4-74f4d13427dc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:32.751955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 562/1110] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:31:32 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:31:34.736047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c782-3668-4e82-a402-49aeb1efd953 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:34.736737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c782-3668-4e82-a402-49aeb1efd953 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:34.736737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c782-3668-4e82-a402-49aeb1efd953 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:34.737978 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 549/1111] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:34.743767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12205676-1d39-466f-b18d-7bf53489c42f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:34.744178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12205676-1d39-466f-b18d-7bf53489c42f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:34.744314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12205676-1d39-466f-b18d-7bf53489c42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:34.762158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-12205676-1d39-466f-b18d-7bf53489c42f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:34.763327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-12205676-1d39-466f-b18d-7bf53489c42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:34.817897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:34.829975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 563/1112] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:35.095628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362c782-3668-4e82-a402-49aeb1efd953 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:35.197139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:35.201448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362c782-3668-4e82-a402-49aeb1efd953 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:35.237167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968f8cc-90de-4e05-93d7-5849acadcfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:35.237792 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968f8cc-90de-4e05-93d7-5849acadcfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:35.238075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968f8cc-90de-4e05-93d7-5849acadcfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:35.238361 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968f8cc-90de-4e05-93d7-5849acadcfc6 None admin] PATH_INFO: `/v3/roles/6f9c4ad710e84eaabc415a1dd1541055` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:35.239742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968f8cc-90de-4e05-93d7-5849acadcfc6 None admin] RBAC: Authorizing `identity:get_role(role_id=6f9c4ad710e84eaabc415a1dd1541055)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:35.241136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968f8cc-90de-4e05-93d7-5849acadcfc6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:35.242454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 550/1113] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:35 2022] GET /identity/v3/roles/6f9c4ad710e84eaabc415a1dd1541055 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:35.248264 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12205676-1d39-466f-b18d-7bf53489c42f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:35.347893 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12205676-1d39-466f-b18d-7bf53489c42f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:35.382274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d74ceb-afcc-43f0-902e-9e049a9bc0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:35.382870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d74ceb-afcc-43f0-902e-9e049a9bc0af None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:35.383152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d74ceb-afcc-43f0-902e-9e049a9bc0af None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:35.383425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d74ceb-afcc-43f0-902e-9e049a9bc0af None admin] PATH_INFO: `/v3/users/27e8124fdb9949229618a8a3b5da0ff8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:35.396658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d74ceb-afcc-43f0-902e-9e049a9bc0af None admin] RBAC: Authorizing `identity:get_user(user_id=27e8124fdb9949229618a8a3b5da0ff8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:35.398083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d74ceb-afcc-43f0-902e-9e049a9bc0af None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:35.400802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 564/1114] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:35 2022] GET /identity/v3/users/27e8124fdb9949229618a8a3b5da0ff8 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:35.406499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968f8cc-90de-4e05-93d7-5849acadcfc6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:35.503077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968f8cc-90de-4e05-93d7-5849acadcfc6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:35.536035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c536e09-76c7-44ce-b2c9-e44584756204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:35.536654 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c536e09-76c7-44ce-b2c9-e44584756204 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:35.536936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c536e09-76c7-44ce-b2c9-e44584756204 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:35.537208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c536e09-76c7-44ce-b2c9-e44584756204 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:35.570095 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:35.602742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c536e09-76c7-44ce-b2c9-e44584756204 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:35.604090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c536e09-76c7-44ce-b2c9-e44584756204 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:35.702033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 551/1115] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:31:35.707941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d74ceb-afcc-43f0-902e-9e049a9bc0af None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:35.756440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:35.806761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d74ceb-afcc-43f0-902e-9e049a9bc0af None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:35.839541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cff216-cbd4-460b-ab3f-2be0aadcf8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:35.840152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cff216-cbd4-460b-ab3f-2be0aadcf8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:35.840432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cff216-cbd4-460b-ab3f-2be0aadcf8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:35.840732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cff216-cbd4-460b-ab3f-2be0aadcf8e2 None admin] PATH_INFO: `/v3/projects/5ca780a28b784f98a577c8e249cd0f59` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:35.841835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cff216-cbd4-460b-ab3f-2be0aadcf8e2 None admin] RBAC: Authorizing `identity:get_project(project_id=5ca780a28b784f98a577c8e249cd0f59)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:35.843241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cff216-cbd4-460b-ab3f-2be0aadcf8e2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:35.844883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 565/1116] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:31:35 2022] GET /identity/v3/projects/5ca780a28b784f98a577c8e249cd0f59 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:35.850896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c536e09-76c7-44ce-b2c9-e44584756204 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:35.914633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:35.948167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c536e09-76c7-44ce-b2c9-e44584756204 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:35.981492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152586a-a425-4ea6-8f73-cb684d3109ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:35.982082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152586a-a425-4ea6-8f73-cb684d3109ec None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:35.982362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152586a-a425-4ea6-8f73-cb684d3109ec None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:35.982639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152586a-a425-4ea6-8f73-cb684d3109ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:35.983350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152586a-a425-4ea6-8f73-cb684d3109ec None admin] RBAC: Adding query filter params (role.id=6f9c4ad710e84eaabc415a1dd1541055, scope.project.id=5ca780a28b784f98a577c8e249cd0f59, user.id=27e8124fdb9949229618a8a3b5da0ff8) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:35.983655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152586a-a425-4ea6-8f73-cb684d3109ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:35.985099 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152586a-a425-4ea6-8f73-cb684d3109ec None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:35.990201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 552/1117] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:31:35 2022] GET /identity/v3/role_assignments?user.id=27e8124fdb9949229618a8a3b5da0ff8&scope.project.id=5ca780a28b784f98a577c8e249cd0f59&role.id=6f9c4ad710e84eaabc415a1dd1541055 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:37.950293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fddab3-9da7-420a-9945-38d908730e6d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:37.950799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fddab3-9da7-420a-9945-38d908730e6d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:37.950799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fddab3-9da7-420a-9945-38d908730e6d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:37.952093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 566/1118] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:37.957238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86fe33-0f46-4f6b-a2ec-042fb22c3a21 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:37.957640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86fe33-0f46-4f6b-a2ec-042fb22c3a21 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:37.957795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86fe33-0f46-4f6b-a2ec-042fb22c3a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:37.975693 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1b86fe33-0f46-4f6b-a2ec-042fb22c3a21 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:37.976906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1b86fe33-0f46-4f6b-a2ec-042fb22c3a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:38.041965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 553/1119] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:38.311713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fddab3-9da7-420a-9945-38d908730e6d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:38.409756 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fddab3-9da7-420a-9945-38d908730e6d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:38.426044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:38.443969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ea54a-3346-4924-8676-b8ea7940fad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:38.444603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ea54a-3346-4924-8676-b8ea7940fad8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:38.444881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ea54a-3346-4924-8676-b8ea7940fad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:38.445147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ea54a-3346-4924-8676-b8ea7940fad8 None admin] PATH_INFO: `/v3/roles/6f9c4ad710e84eaabc415a1dd1541055` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:38.446626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ea54a-3346-4924-8676-b8ea7940fad8 None admin] RBAC: Authorizing `identity:get_role(role_id=6f9c4ad710e84eaabc415a1dd1541055)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:38.447963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ea54a-3346-4924-8676-b8ea7940fad8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:38.449281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 567/1120] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:38 2022] GET /identity/v3/roles/6f9c4ad710e84eaabc415a1dd1541055 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:38.455095 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86fe33-0f46-4f6b-a2ec-042fb22c3a21 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:38.516016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:38.554446 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86fe33-0f46-4f6b-a2ec-042fb22c3a21 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:38.588364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684745d-ffa2-4282-9174-95fc6f1d8bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:38.589013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684745d-ffa2-4282-9174-95fc6f1d8bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:38.589299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684745d-ffa2-4282-9174-95fc6f1d8bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:38.589575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684745d-ffa2-4282-9174-95fc6f1d8bb5 None admin] PATH_INFO: `/v3/users/27e8124fdb9949229618a8a3b5da0ff8` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:38.590698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684745d-ffa2-4282-9174-95fc6f1d8bb5 None admin] RBAC: Authorizing `identity:get_user(user_id=27e8124fdb9949229618a8a3b5da0ff8)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:38.592133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684745d-ffa2-4282-9174-95fc6f1d8bb5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:38.593728 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 554/1121] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:38 2022] GET /identity/v3/users/27e8124fdb9949229618a8a3b5da0ff8 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:38.599390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ea54a-3346-4924-8676-b8ea7940fad8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:38.697189 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ea54a-3346-4924-8676-b8ea7940fad8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:38.729525 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff9aa9-7ef3-4be8-8110-95d1cacdc4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:38.730131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff9aa9-7ef3-4be8-8110-95d1cacdc4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:38.730415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff9aa9-7ef3-4be8-8110-95d1cacdc4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:38.730685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff9aa9-7ef3-4be8-8110-95d1cacdc4b3 None admin] PATH_INFO: `/v3/projects/5ca780a28b784f98a577c8e249cd0f59` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:38.731776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff9aa9-7ef3-4be8-8110-95d1cacdc4b3 None admin] RBAC: Authorizing `identity:get_project(project_id=5ca780a28b784f98a577c8e249cd0f59)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:38.733220 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff9aa9-7ef3-4be8-8110-95d1cacdc4b3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:38.734822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 568/1122] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:31:38 2022] GET /identity/v3/projects/5ca780a28b784f98a577c8e249cd0f59 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:38.740729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684745d-ffa2-4282-9174-95fc6f1d8bb5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:38.836558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684745d-ffa2-4282-9174-95fc6f1d8bb5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:38.864190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:38.870204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515fd47-e7a8-4870-8cda-85c59d3c255b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:38.870851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515fd47-e7a8-4870-8cda-85c59d3c255b None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:38.871135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515fd47-e7a8-4870-8cda-85c59d3c255b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:38.871411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515fd47-e7a8-4870-8cda-85c59d3c255b None admin] PATH_INFO: `/v3/projects/5ca780a28b784f98a577c8e249cd0f59/users/27e8124fdb9949229618a8a3b5da0ff8/roles/6f9c4ad710e84eaabc415a1dd1541055` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:38.873806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515fd47-e7a8-4870-8cda-85c59d3c255b None admin] RBAC: Authorizing `identity:create_grant(project_id=5ca780a28b784f98a577c8e249cd0f59, user_id=27e8124fdb9949229618a8a3b5da0ff8, role_id=6f9c4ad710e84eaabc415a1dd1541055)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:38.875234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515fd47-e7a8-4870-8cda-85c59d3c255b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:38.887959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 555/1123] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:31:38 2022] PUT /identity/v3/projects/5ca780a28b784f98a577c8e249cd0f59/users/27e8124fdb9949229618a8a3b5da0ff8/roles/6f9c4ad710e84eaabc415a1dd1541055 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:31:40.834788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6526b3-1c67-4d7b-8fd0-cb32448a6443 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:40.835365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6526b3-1c67-4d7b-8fd0-cb32448a6443 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:40.835365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6526b3-1c67-4d7b-8fd0-cb32448a6443 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:40.837257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 569/1124] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:40.842434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62ef9e-5ed2-48b5-a4a1-f132356c5479 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:40.842875 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62ef9e-5ed2-48b5-a4a1-f132356c5479 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:40.842999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62ef9e-5ed2-48b5-a4a1-f132356c5479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:40.861257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3c62ef9e-5ed2-48b5-a4a1-f132356c5479 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:40.862449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3c62ef9e-5ed2-48b5-a4a1-f132356c5479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:40.927336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 556/1125] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:41.205495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6526b3-1c67-4d7b-8fd0-cb32448a6443 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:41.260900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:41.309229 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6526b3-1c67-4d7b-8fd0-cb32448a6443 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:41.342408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d31aa3-d732-49fb-86bd-07e832fd50d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:41.343019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d31aa3-d732-49fb-86bd-07e832fd50d0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:41.343296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d31aa3-d732-49fb-86bd-07e832fd50d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:41.343562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d31aa3-d732-49fb-86bd-07e832fd50d0 None admin] PATH_INFO: `/v3/roles/6f9c4ad710e84eaabc415a1dd1541055` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:41.344938 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d31aa3-d732-49fb-86bd-07e832fd50d0 None admin] RBAC: Authorizing `identity:get_role(role_id=6f9c4ad710e84eaabc415a1dd1541055)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:41.346302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d31aa3-d732-49fb-86bd-07e832fd50d0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:41.347644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 570/1126] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:41 2022] GET /identity/v3/roles/6f9c4ad710e84eaabc415a1dd1541055 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:41.355334 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62ef9e-5ed2-48b5-a4a1-f132356c5479 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:41.452592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62ef9e-5ed2-48b5-a4a1-f132356c5479 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:41.471934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:41.486695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0417e47-d58d-4bca-9254-c04739a37ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:41.487315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0417e47-d58d-4bca-9254-c04739a37ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:41.487597 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0417e47-d58d-4bca-9254-c04739a37ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:41.487873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0417e47-d58d-4bca-9254-c04739a37ea5 None admin] PATH_INFO: `/v3/users/27e8124fdb9949229618a8a3b5da0ff8` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:41.489016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0417e47-d58d-4bca-9254-c04739a37ea5 None admin] RBAC: Authorizing `identity:get_user(user_id=27e8124fdb9949229618a8a3b5da0ff8)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:41.490444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0417e47-d58d-4bca-9254-c04739a37ea5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:41.492007 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 557/1127] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:41 2022] GET /identity/v3/users/27e8124fdb9949229618a8a3b5da0ff8 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:41.498932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d31aa3-d732-49fb-86bd-07e832fd50d0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:41.600603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d31aa3-d732-49fb-86bd-07e832fd50d0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:41.620571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:41.635614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3e4b0-9f6e-45b1-a039-98b0550f3efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:41.636209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3e4b0-9f6e-45b1-a039-98b0550f3efe None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:41.636512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3e4b0-9f6e-45b1-a039-98b0550f3efe None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:41.636796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3e4b0-9f6e-45b1-a039-98b0550f3efe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:41.705207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3e4b0-9f6e-45b1-a039-98b0550f3efe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:41.706646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3e4b0-9f6e-45b1-a039-98b0550f3efe None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:41.803359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 571/1128] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:31:41.809402 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0417e47-d58d-4bca-9254-c04739a37ea5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:41.906572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0417e47-d58d-4bca-9254-c04739a37ea5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:41.939853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383d34b-daf4-4b6c-a4c0-2f48a862b5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:41.940463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383d34b-daf4-4b6c-a4c0-2f48a862b5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:41.940773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383d34b-daf4-4b6c-a4c0-2f48a862b5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:41.941049 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383d34b-daf4-4b6c-a4c0-2f48a862b5e4 None admin] PATH_INFO: `/v3/projects/5ca780a28b784f98a577c8e249cd0f59` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:41.942443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383d34b-daf4-4b6c-a4c0-2f48a862b5e4 None admin] RBAC: Authorizing `identity:get_project(project_id=5ca780a28b784f98a577c8e249cd0f59)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:41.943862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383d34b-daf4-4b6c-a4c0-2f48a862b5e4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:41.945489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 558/1129] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:31:41 2022] GET /identity/v3/projects/5ca780a28b784f98a577c8e249cd0f59 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:41.952036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3e4b0-9f6e-45b1-a039-98b0550f3efe None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:41.973582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:42.051069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3e4b0-9f6e-45b1-a039-98b0550f3efe None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:42.085076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb90ad7-ffe1-445b-8231-0ff93bd55f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:42.085661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb90ad7-ffe1-445b-8231-0ff93bd55f36 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:42.085939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb90ad7-ffe1-445b-8231-0ff93bd55f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:42.086208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb90ad7-ffe1-445b-8231-0ff93bd55f36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:42.086963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb90ad7-ffe1-445b-8231-0ff93bd55f36 None admin] RBAC: Adding query filter params (role.id=6f9c4ad710e84eaabc415a1dd1541055, scope.project.id=5ca780a28b784f98a577c8e249cd0f59, user.id=27e8124fdb9949229618a8a3b5da0ff8) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:42.087265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb90ad7-ffe1-445b-8231-0ff93bd55f36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:42.088715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb90ad7-ffe1-445b-8231-0ff93bd55f36 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:42.094577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 572/1130] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:31:41 2022] GET /identity/v3/role_assignments?user.id=27e8124fdb9949229618a8a3b5da0ff8&scope.project.id=5ca780a28b784f98a577c8e249cd0f59&role.id=6f9c4ad710e84eaabc415a1dd1541055 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:44.033154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd9c1c-62c1-4f10-88b6-6a2abb515c15 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:44.033687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd9c1c-62c1-4f10-88b6-6a2abb515c15 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:44.033687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd9c1c-62c1-4f10-88b6-6a2abb515c15 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:44.034961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 559/1131] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:44.041140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73651ae-50b1-4f0c-a206-0b8dc95e6383 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:44.041578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73651ae-50b1-4f0c-a206-0b8dc95e6383 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:44.041726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73651ae-50b1-4f0c-a206-0b8dc95e6383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:44.060420 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a73651ae-50b1-4f0c-a206-0b8dc95e6383 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:44.061631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a73651ae-50b1-4f0c-a206-0b8dc95e6383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:44.130993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 573/1132] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:44.400184 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd9c1c-62c1-4f10-88b6-6a2abb515c15 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:44.456921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:44.501142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd9c1c-62c1-4f10-88b6-6a2abb515c15 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:44.534567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a513d-22db-44c6-91fc-d9821d77b005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:44.535190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a513d-22db-44c6-91fc-d9821d77b005 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:44.535472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a513d-22db-44c6-91fc-d9821d77b005 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:44.535748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a513d-22db-44c6-91fc-d9821d77b005 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:44.536890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a513d-22db-44c6-91fc-d9821d77b005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:44.538336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a513d-22db-44c6-91fc-d9821d77b005 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:44.540432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 560/1133] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:44.546265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73651ae-50b1-4f0c-a206-0b8dc95e6383 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:44.561456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:44.644248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73651ae-50b1-4f0c-a206-0b8dc95e6383 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:44.678194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555cf8c-5d2b-4030-bcac-7e938821f754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:44.678327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555cf8c-5d2b-4030-bcac-7e938821f754 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:44.678505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555cf8c-5d2b-4030-bcac-7e938821f754 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:44.678777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555cf8c-5d2b-4030-bcac-7e938821f754 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:44.679419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555cf8c-5d2b-4030-bcac-7e938821f754 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:44.682319 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555cf8c-5d2b-4030-bcac-7e938821f754 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:44.709253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 574/1134] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:31:44 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:46.647842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e370e-f659-40c3-b3ec-683661f1ea27 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:46.648404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e370e-f659-40c3-b3ec-683661f1ea27 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:46.648404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e370e-f659-40c3-b3ec-683661f1ea27 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:46.650570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 561/1135] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:46.656681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55179bf1-2777-428b-a7a7-15c669e29918 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:46.657151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55179bf1-2777-428b-a7a7-15c669e29918 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:46.657247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55179bf1-2777-428b-a7a7-15c669e29918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:46.678140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-55179bf1-2777-428b-a7a7-15c669e29918 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:46.679209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-55179bf1-2777-428b-a7a7-15c669e29918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:46.738566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:46.748613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 575/1136] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:47.030089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e370e-f659-40c3-b3ec-683661f1ea27 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:47.128545 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e370e-f659-40c3-b3ec-683661f1ea27 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:47.151464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:47.163938 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b512c-c04e-4d68-b9a0-d5ab74facc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:47.165132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b512c-c04e-4d68-b9a0-d5ab74facc51 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:47.165583 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b512c-c04e-4d68-b9a0-d5ab74facc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:47.165864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b512c-c04e-4d68-b9a0-d5ab74facc51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:47.167003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b512c-c04e-4d68-b9a0-d5ab74facc51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:47.168445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b512c-c04e-4d68-b9a0-d5ab74facc51 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:47.170040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 562/1137] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:31:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:47.175815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55179bf1-2777-428b-a7a7-15c669e29918 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:47.273297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55179bf1-2777-428b-a7a7-15c669e29918 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:47.310890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7b577-1367-4bbf-94f7-70b45e477824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:47.311419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7b577-1367-4bbf-94f7-70b45e477824 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:47.311699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7b577-1367-4bbf-94f7-70b45e477824 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:47.311971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7b577-1367-4bbf-94f7-70b45e477824 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:47.312650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7b577-1367-4bbf-94f7-70b45e477824 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:47.314070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7b577-1367-4bbf-94f7-70b45e477824 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:47.335462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 576/1138] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:31:47 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:49.264285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c67e-9524-437f-bd22-0769b499aeb7 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:49.264863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c67e-9524-437f-bd22-0769b499aeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:49.265013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c67e-9524-437f-bd22-0769b499aeb7 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:49.266358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 563/1139] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:49.271523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3dd0a-9fe1-4203-87ec-e37bd76d1822 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:49.271965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3dd0a-9fe1-4203-87ec-e37bd76d1822 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:49.272102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3dd0a-9fe1-4203-87ec-e37bd76d1822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:49.291308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0af3dd0a-9fe1-4203-87ec-e37bd76d1822 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:49.292481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0af3dd0a-9fe1-4203-87ec-e37bd76d1822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:49.338414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:49.358423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 577/1140] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:49.621320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557c67e-9524-437f-bd22-0769b499aeb7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:49.723940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557c67e-9524-437f-bd22-0769b499aeb7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:49.757082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c7db-243a-4be4-a9d9-0eda99b17bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:49.757700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c7db-243a-4be4-a9d9-0eda99b17bae None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:49.757980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c7db-243a-4be4-a9d9-0eda99b17bae None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:49.758259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c7db-243a-4be4-a9d9-0eda99b17bae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:49.768735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22c7db-243a-4be4-a9d9-0eda99b17bae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-6d22c7db-243a-4be4-a9d9-0eda99b17bae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:49.773499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:49.774681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:49.775767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 564/1141] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:31:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:49.781382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3dd0a-9fe1-4203-87ec-e37bd76d1822 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:49.876859 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3dd0a-9fe1-4203-87ec-e37bd76d1822 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:49.910533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7808b-68e8-4c13-a827-c4eda007b5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:49.911129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7808b-68e8-4c13-a827-c4eda007b5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:49.911410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7808b-68e8-4c13-a827-c4eda007b5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:49.911683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7808b-68e8-4c13-a827-c4eda007b5f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:49.912339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7808b-68e8-4c13-a827-c4eda007b5f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:49.912675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7808b-68e8-4c13-a827-c4eda007b5f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:49.914032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7808b-68e8-4c13-a827-c4eda007b5f4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:49.921945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 578/1142] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:31:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:49.927695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c7db-243a-4be4-a9d9-0eda99b17bae None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:49.981879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:50.028898 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c7db-243a-4be4-a9d9-0eda99b17bae None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:50.061621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7307f-7170-4741-8cf7-add81898286c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:50.062234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7307f-7170-4741-8cf7-add81898286c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:50.062516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7307f-7170-4741-8cf7-add81898286c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:50.062796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7307f-7170-4741-8cf7-add81898286c None admin] PATH_INFO: `/v3/users/099e33324de24067962b73aed4709a5f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:50.076443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7307f-7170-4741-8cf7-add81898286c None admin] RBAC: Authorizing `identity:get_user(user_id=099e33324de24067962b73aed4709a5f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:50.077902 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7307f-7170-4741-8cf7-add81898286c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:50.079465 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 565/1143] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:49 2022] GET /identity/v3/users/099e33324de24067962b73aed4709a5f => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:50.085333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7808b-68e8-4c13-a827-c4eda007b5f4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:50.131407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:50.185032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7808b-68e8-4c13-a827-c4eda007b5f4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:50.217819 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ae5f5-ea66-4c84-9654-490f3a1810d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:50.218415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae5f5-ea66-4c84-9654-490f3a1810d7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:50.218690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae5f5-ea66-4c84-9654-490f3a1810d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:50.218958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae5f5-ea66-4c84-9654-490f3a1810d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:50.282675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ae5f5-ea66-4c84-9654-490f3a1810d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:50.284094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ae5f5-ea66-4c84-9654-490f3a1810d7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:50.326452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:50.380090 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 579/1144] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:31:50.386835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7307f-7170-4741-8cf7-add81898286c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:50.484540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7307f-7170-4741-8cf7-add81898286c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:50.495383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:50.518874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a252e34-e812-45a6-99ff-f79299681140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:50.519493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a252e34-e812-45a6-99ff-f79299681140 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:50.519779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a252e34-e812-45a6-99ff-f79299681140 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:50.520054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a252e34-e812-45a6-99ff-f79299681140 None admin] PATH_INFO: `/v3/projects/36c9f29e9ec4452b9b34b2b9da6ed06d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:50.521172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a252e34-e812-45a6-99ff-f79299681140 None admin] RBAC: Authorizing `identity:get_project(project_id=36c9f29e9ec4452b9b34b2b9da6ed06d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:50.522594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a252e34-e812-45a6-99ff-f79299681140 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:50.524248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 566/1145] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:31:50 2022] GET /identity/v3/projects/36c9f29e9ec4452b9b34b2b9da6ed06d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:50.530377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ae5f5-ea66-4c84-9654-490f3a1810d7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:50.627276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ae5f5-ea66-4c84-9654-490f3a1810d7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:50.660788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f17c2f-d7d9-41b5-9fcc-32959cc64ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:50.661319 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f17c2f-d7d9-41b5-9fcc-32959cc64ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:50.661602 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f17c2f-d7d9-41b5-9fcc-32959cc64ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:50.661877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f17c2f-d7d9-41b5-9fcc-32959cc64ac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:50.662589 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f17c2f-d7d9-41b5-9fcc-32959cc64ac8 None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=36c9f29e9ec4452b9b34b2b9da6ed06d, user.id=099e33324de24067962b73aed4709a5f) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:50.662896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f17c2f-d7d9-41b5-9fcc-32959cc64ac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:50.664305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f17c2f-d7d9-41b5-9fcc-32959cc64ac8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:50.669669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 580/1146] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:31:50 2022] GET /identity/v3/role_assignments?user.id=099e33324de24067962b73aed4709a5f&scope.project.id=36c9f29e9ec4452b9b34b2b9da6ed06d&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:52.573746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5fd46-2d34-4c5a-af97-4fbc6c636bec None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:52.574321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5fd46-2d34-4c5a-af97-4fbc6c636bec None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:52.574321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5fd46-2d34-4c5a-af97-4fbc6c636bec None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:52.575642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 567/1147] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:52.580887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba8877-d33a-4d6b-8542-09b9d61f3a8f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:52.581342 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba8877-d33a-4d6b-8542-09b9d61f3a8f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:52.581446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba8877-d33a-4d6b-8542-09b9d61f3a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:52.599749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-76ba8877-d33a-4d6b-8542-09b9d61f3a8f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:52.600951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-76ba8877-d33a-4d6b-8542-09b9d61f3a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:52.603472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:52.673265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 581/1148] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:52.950463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5fd46-2d34-4c5a-af97-4fbc6c636bec None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:53.048848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5fd46-2d34-4c5a-af97-4fbc6c636bec None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:53.081153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeacf5d-73e5-4890-8021-307e498bb995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:53.081759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeacf5d-73e5-4890-8021-307e498bb995 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:53.082035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeacf5d-73e5-4890-8021-307e498bb995 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:53.082306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeacf5d-73e5-4890-8021-307e498bb995 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:53.092004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeacf5d-73e5-4890-8021-307e498bb995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-adeacf5d-73e5-4890-8021-307e498bb995 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:53.096796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:53.097879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:53.099061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 568/1149] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:31:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:53.104796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba8877-d33a-4d6b-8542-09b9d61f3a8f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:53.201626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba8877-d33a-4d6b-8542-09b9d61f3a8f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:53.234370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2b1a4-5ed9-4b1c-8688-919b2036e9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:53.234962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2b1a4-5ed9-4b1c-8688-919b2036e9eb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:53.235239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2b1a4-5ed9-4b1c-8688-919b2036e9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:53.235508 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2b1a4-5ed9-4b1c-8688-919b2036e9eb None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:53.236155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2b1a4-5ed9-4b1c-8688-919b2036e9eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:53.236457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2b1a4-5ed9-4b1c-8688-919b2036e9eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:53.237836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2b1a4-5ed9-4b1c-8688-919b2036e9eb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:53.243246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:53.247085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 582/1150] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:31:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:53.252450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeacf5d-73e5-4890-8021-307e498bb995 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:53.293086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:53.352367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeacf5d-73e5-4890-8021-307e498bb995 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:53.385274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1badab7-2bc8-45b0-a922-8518538629da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:53.385891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1badab7-2bc8-45b0-a922-8518538629da None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:53.386174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1badab7-2bc8-45b0-a922-8518538629da None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:53.386454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1badab7-2bc8-45b0-a922-8518538629da None admin] PATH_INFO: `/v3/users/099e33324de24067962b73aed4709a5f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:53.387570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1badab7-2bc8-45b0-a922-8518538629da None admin] RBAC: Authorizing `identity:get_user(user_id=099e33324de24067962b73aed4709a5f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:53.389111 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1badab7-2bc8-45b0-a922-8518538629da None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:53.390683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 569/1151] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:53 2022] GET /identity/v3/users/099e33324de24067962b73aed4709a5f => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:53.396322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2b1a4-5ed9-4b1c-8688-919b2036e9eb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:53.494543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2b1a4-5ed9-4b1c-8688-919b2036e9eb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:53.528570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a10be2-32e8-423b-8559-f10d50ca2bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:53.529193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a10be2-32e8-423b-8559-f10d50ca2bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:53.529482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a10be2-32e8-423b-8559-f10d50ca2bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:53.529758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a10be2-32e8-423b-8559-f10d50ca2bf0 None admin] PATH_INFO: `/v3/projects/36c9f29e9ec4452b9b34b2b9da6ed06d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:53.530872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a10be2-32e8-423b-8559-f10d50ca2bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=36c9f29e9ec4452b9b34b2b9da6ed06d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:53.532286 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a10be2-32e8-423b-8559-f10d50ca2bf0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:53.533924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 583/1152] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:31:53 2022] GET /identity/v3/projects/36c9f29e9ec4452b9b34b2b9da6ed06d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:53.540103 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1badab7-2bc8-45b0-a922-8518538629da None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:53.635011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1badab7-2bc8-45b0-a922-8518538629da None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:53.641973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:53.668431 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111a62b-5805-4528-aaa3-c8688ef35a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:53.669091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111a62b-5805-4528-aaa3-c8688ef35a41 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:53.669368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111a62b-5805-4528-aaa3-c8688ef35a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:53.669641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111a62b-5805-4528-aaa3-c8688ef35a41 None admin] PATH_INFO: `/v3/projects/36c9f29e9ec4452b9b34b2b9da6ed06d/users/099e33324de24067962b73aed4709a5f/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:53.671499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111a62b-5805-4528-aaa3-c8688ef35a41 None admin] RBAC: Authorizing `identity:create_grant(project_id=36c9f29e9ec4452b9b34b2b9da6ed06d, user_id=099e33324de24067962b73aed4709a5f, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:53.672953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111a62b-5805-4528-aaa3-c8688ef35a41 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:53.685674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 570/1153] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:31:53 2022] PUT /identity/v3/projects/36c9f29e9ec4452b9b34b2b9da6ed06d/users/099e33324de24067962b73aed4709a5f/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:31:55.546689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa77c93-3d47-481a-8ba6-52b7d05f71c3 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:55.547246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa77c93-3d47-481a-8ba6-52b7d05f71c3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:55.547246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa77c93-3d47-481a-8ba6-52b7d05f71c3 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:55.548588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 584/1154] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:55.553640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7232f3-8b16-450d-98be-bfd48b578985 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:55.554047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7232f3-8b16-450d-98be-bfd48b578985 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:55.554191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7232f3-8b16-450d-98be-bfd48b578985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:55.573240 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1d7232f3-8b16-450d-98be-bfd48b578985 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:55.574412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1d7232f3-8b16-450d-98be-bfd48b578985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:55.639624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 571/1155] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:31:55.917154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa77c93-3d47-481a-8ba6-52b7d05f71c3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:55.986073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:56.020897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa77c93-3d47-481a-8ba6-52b7d05f71c3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:56.054845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b0d08-53ca-4a81-8e8a-b96cf9f2b450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:56.056563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b0d08-53ca-4a81-8e8a-b96cf9f2b450 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:56.056845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b0d08-53ca-4a81-8e8a-b96cf9f2b450 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:56.057117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b0d08-53ca-4a81-8e8a-b96cf9f2b450 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:56.066979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b0d08-53ca-4a81-8e8a-b96cf9f2b450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e25b0d08-53ca-4a81-8e8a-b96cf9f2b450 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:31:56.073111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:56.074193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:31:56.075238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 585/1156] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:31:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:56.080229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7232f3-8b16-450d-98be-bfd48b578985 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:56.178261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:56.185465 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7232f3-8b16-450d-98be-bfd48b578985 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:56.220053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab498062-e192-4480-ac7f-c0bde9090fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:56.220675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab498062-e192-4480-ac7f-c0bde9090fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:56.220967 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab498062-e192-4480-ac7f-c0bde9090fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:56.221244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab498062-e192-4480-ac7f-c0bde9090fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:56.221912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab498062-e192-4480-ac7f-c0bde9090fa3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:56.222220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab498062-e192-4480-ac7f-c0bde9090fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:56.223590 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab498062-e192-4480-ac7f-c0bde9090fa3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:56.231909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 572/1157] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:31:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:56.237745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b0d08-53ca-4a81-8e8a-b96cf9f2b450 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:56.335799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b0d08-53ca-4a81-8e8a-b96cf9f2b450 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:56.362712 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:56.370737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7095314-5ee6-459c-96c3-14d915f1d819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:56.371354 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7095314-5ee6-459c-96c3-14d915f1d819 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:56.371634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7095314-5ee6-459c-96c3-14d915f1d819 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:56.371904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7095314-5ee6-459c-96c3-14d915f1d819 None admin] PATH_INFO: `/v3/users/099e33324de24067962b73aed4709a5f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:56.373033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7095314-5ee6-459c-96c3-14d915f1d819 None admin] RBAC: Authorizing `identity:get_user(user_id=099e33324de24067962b73aed4709a5f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:56.374451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7095314-5ee6-459c-96c3-14d915f1d819 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:56.376007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 586/1158] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:31:56 2022] GET /identity/v3/users/099e33324de24067962b73aed4709a5f => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:56.381910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab498062-e192-4480-ac7f-c0bde9090fa3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:56.481417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab498062-e192-4480-ac7f-c0bde9090fa3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:56.513605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9b204-e20c-4c94-9faf-b73cade51892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:56.514193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9b204-e20c-4c94-9faf-b73cade51892 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:56.514474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9b204-e20c-4c94-9faf-b73cade51892 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:56.514747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9b204-e20c-4c94-9faf-b73cade51892 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:56.541304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:56.581342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c9b204-e20c-4c94-9faf-b73cade51892 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:56.582856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c9b204-e20c-4c94-9faf-b73cade51892 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:56.676804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 573/1159] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:31:56.683901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7095314-5ee6-459c-96c3-14d915f1d819 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:56.780614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7095314-5ee6-459c-96c3-14d915f1d819 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:56.813140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de4e63-e21a-4c49-90f5-54fbb1dfdf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:56.813747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de4e63-e21a-4c49-90f5-54fbb1dfdf82 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:56.814025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de4e63-e21a-4c49-90f5-54fbb1dfdf82 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:56.814297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de4e63-e21a-4c49-90f5-54fbb1dfdf82 None admin] PATH_INFO: `/v3/projects/36c9f29e9ec4452b9b34b2b9da6ed06d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:56.815389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de4e63-e21a-4c49-90f5-54fbb1dfdf82 None admin] RBAC: Authorizing `identity:get_project(project_id=36c9f29e9ec4452b9b34b2b9da6ed06d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:56.816831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de4e63-e21a-4c49-90f5-54fbb1dfdf82 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:56.818696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 587/1160] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:31:56 2022] GET /identity/v3/projects/36c9f29e9ec4452b9b34b2b9da6ed06d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:56.825547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9b204-e20c-4c94-9faf-b73cade51892 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:56.881664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:56.922297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9b204-e20c-4c94-9faf-b73cade51892 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:56.955695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23bd34-1108-4993-9768-bb6b9f065289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:56.956281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23bd34-1108-4993-9768-bb6b9f065289 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:56.956605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23bd34-1108-4993-9768-bb6b9f065289 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:56.956866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23bd34-1108-4993-9768-bb6b9f065289 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:56.957574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23bd34-1108-4993-9768-bb6b9f065289 None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=36c9f29e9ec4452b9b34b2b9da6ed06d, user.id=099e33324de24067962b73aed4709a5f) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:56.957881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23bd34-1108-4993-9768-bb6b9f065289 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:56.959292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23bd34-1108-4993-9768-bb6b9f065289 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:56.964995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 574/1161] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:31:56 2022] GET /identity/v3/role_assignments?user.id=099e33324de24067962b73aed4709a5f&scope.project.id=36c9f29e9ec4452b9b34b2b9da6ed06d&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:31:58.967453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028dd5a-6dd5-4b82-8fa5-e52028216baf None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:58.967952 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028dd5a-6dd5-4b82-8fa5-e52028216baf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:58.967952 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028dd5a-6dd5-4b82-8fa5-e52028216baf None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:58.969277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 588/1162] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:31:58.974505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7946f6-2260-439f-aaa0-55d67539588f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:58.974949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7946f6-2260-439f-aaa0-55d67539588f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:58.975041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7946f6-2260-439f-aaa0-55d67539588f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:58.994130 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2c7946f6-2260-439f-aaa0-55d67539588f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:58.995311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2c7946f6-2260-439f-aaa0-55d67539588f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:59.060087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 575/1163] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:31:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:31:59.333651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028dd5a-6dd5-4b82-8fa5-e52028216baf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:59.395855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:59.434305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028dd5a-6dd5-4b82-8fa5-e52028216baf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:59.467408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448361f-a80b-4bb7-b33e-5b4fabe49026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:59.467998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448361f-a80b-4bb7-b33e-5b4fabe49026 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:59.468276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448361f-a80b-4bb7-b33e-5b4fabe49026 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:59.468572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448361f-a80b-4bb7-b33e-5b4fabe49026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:59.534207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448361f-a80b-4bb7-b33e-5b4fabe49026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:59.535637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448361f-a80b-4bb7-b33e-5b4fabe49026 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:59.594389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:59.631355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 589/1164] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:31:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:31:59.638539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7946f6-2260-439f-aaa0-55d67539588f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:59.688626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:59.734589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7946f6-2260-439f-aaa0-55d67539588f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:59.766878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff423792-572a-4682-aff0-90adcda2ffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:59.767499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff423792-572a-4682-aff0-90adcda2ffa5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:59.767776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff423792-572a-4682-aff0-90adcda2ffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:59.768047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff423792-572a-4682-aff0-90adcda2ffa5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:59.774960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff423792-572a-4682-aff0-90adcda2ffa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:59.776382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff423792-572a-4682-aff0-90adcda2ffa5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-ff423792-572a-4682-aff0-90adcda2ffa5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:31:59.787383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:31:59.788580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:31:59.789815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 576/1165] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Aug 31 11:31:59 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:31:59.794532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448361f-a80b-4bb7-b33e-5b4fabe49026 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:59.891298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448361f-a80b-4bb7-b33e-5b4fabe49026 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:59.923963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40205ef2-b883-408a-a6ea-f4786408eac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:59.924583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40205ef2-b883-408a-a6ea-f4786408eac0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:59.924867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40205ef2-b883-408a-a6ea-f4786408eac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:59.925141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40205ef2-b883-408a-a6ea-f4786408eac0 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:59.925783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40205ef2-b883-408a-a6ea-f4786408eac0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:59.926088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40205ef2-b883-408a-a6ea-f4786408eac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:59.927433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40205ef2-b883-408a-a6ea-f4786408eac0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:59.935183 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 590/1166] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Aug 31 11:31:59 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:01.828182 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4e36d-2178-46e0-96df-91331e525ba0 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:01.828714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4e36d-2178-46e0-96df-91331e525ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:01.828818 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4e36d-2178-46e0-96df-91331e525ba0 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:01.830144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 577/1167] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:32:01.835169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc0870-47fc-4642-b2c2-3df0104c3cee None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:01.835541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc0870-47fc-4642-b2c2-3df0104c3cee None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:01.835721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc0870-47fc-4642-b2c2-3df0104c3cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:01.854167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-87dc0870-47fc-4642-b2c2-3df0104c3cee None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:32:01.855346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-87dc0870-47fc-4642-b2c2-3df0104c3cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:32:01.878106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:01.921561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 591/1168] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:32:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:32:02.192927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4e36d-2178-46e0-96df-91331e525ba0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:02.295873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4e36d-2178-46e0-96df-91331e525ba0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:02.309306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:02.331163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46dad2-6454-4683-bf8c-430bea94aac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:02.331760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46dad2-6454-4683-bf8c-430bea94aac3 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:02.332049 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46dad2-6454-4683-bf8c-430bea94aac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:02.332326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46dad2-6454-4683-bf8c-430bea94aac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:02.333026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46dad2-6454-4683-bf8c-430bea94aac3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:02.334413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46dad2-6454-4683-bf8c-430bea94aac3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:02.350634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 578/1169] 10.222.0.60 () {60 vars in 1138 bytes} [Wed Aug 31 11:32:02 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:32:04.259262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3897acc-6aaf-4746-927a-d6b638e6ea93 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:04.259851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3897acc-6aaf-4746-927a-d6b638e6ea93 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:04.259851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3897acc-6aaf-4746-927a-d6b638e6ea93 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:04.261185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 592/1170] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:32:04.266527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80633f35-012e-4c7c-b01d-d2e97de4349a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:04.266945 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80633f35-012e-4c7c-b01d-d2e97de4349a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:04.267089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80633f35-012e-4c7c-b01d-d2e97de4349a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:04.285002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-80633f35-012e-4c7c-b01d-d2e97de4349a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:32:04.286194 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-80633f35-012e-4c7c-b01d-d2e97de4349a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:32:04.351006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 579/1171] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:32:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:32:04.616991 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3897acc-6aaf-4746-927a-d6b638e6ea93 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:04.719111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3897acc-6aaf-4746-927a-d6b638e6ea93 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:04.753225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5d93d-8cd4-4101-80ec-1495e32c3857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:04.753839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5d93d-8cd4-4101-80ec-1495e32c3857 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:04.754118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5d93d-8cd4-4101-80ec-1495e32c3857 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:04.754390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5d93d-8cd4-4101-80ec-1495e32c3857 None admin] PATH_INFO: `/v3/domains/f5781328981f4a88b7f32b01cd64c2e2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:04.764561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5d93d-8cd4-4101-80ec-1495e32c3857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f5781328981f4a88b7f32b01cd64c2e2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:04.765978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5d93d-8cd4-4101-80ec-1495e32c3857 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:04.767558 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 593/1172] 10.222.0.60 () {56 vars in 1183 bytes} [Wed Aug 31 11:32:04 2022] GET /identity/v3/domains/f5781328981f4a88b7f32b01cd64c2e2 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:04.773552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80633f35-012e-4c7c-b01d-d2e97de4349a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:04.869862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80633f35-012e-4c7c-b01d-d2e97de4349a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:04.871267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:04.904504 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f42571-212a-4d34-a962-f5a1855844f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:04.905112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f42571-212a-4d34-a962-f5a1855844f1 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:04.905395 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f42571-212a-4d34-a962-f5a1855844f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:04.905671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f42571-212a-4d34-a962-f5a1855844f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:04.906306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f42571-212a-4d34-a962-f5a1855844f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:04.907745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f42571-212a-4d34-a962-f5a1855844f1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:04.934911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 580/1173] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:32:04 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:32:06.920209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996e755-44df-4da7-83c5-4257ddc8e81d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:06.920870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996e755-44df-4da7-83c5-4257ddc8e81d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:06.920870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996e755-44df-4da7-83c5-4257ddc8e81d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:06.923134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 594/1174] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:32:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:32:06.928561 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686711-f116-4671-9451-e074b519fd78 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:06.928999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686711-f116-4671-9451-e074b519fd78 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:06.929111 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686711-f116-4671-9451-e074b519fd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:06.948299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cf686711-f116-4671-9451-e074b519fd78 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:32:06.949504 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cf686711-f116-4671-9451-e074b519fd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:32:07.013748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 581/1175] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:32:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:32:07.291568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996e755-44df-4da7-83c5-4257ddc8e81d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:07.310631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:07.391149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996e755-44df-4da7-83c5-4257ddc8e81d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:07.422873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeec0de-8ab8-497d-a02a-3025825e2a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:07.423477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeec0de-8ab8-497d-a02a-3025825e2a76 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:07.423750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeec0de-8ab8-497d-a02a-3025825e2a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:07.424015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeec0de-8ab8-497d-a02a-3025825e2a76 None admin] PATH_INFO: `/v3/domains/f5781328981f4a88b7f32b01cd64c2e2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:07.425098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeec0de-8ab8-497d-a02a-3025825e2a76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f5781328981f4a88b7f32b01cd64c2e2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:07.426512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeec0de-8ab8-497d-a02a-3025825e2a76 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:07.428036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 595/1176] 10.222.0.60 () {56 vars in 1183 bytes} [Wed Aug 31 11:32:07 2022] GET /identity/v3/domains/f5781328981f4a88b7f32b01cd64c2e2 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:07.434341 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf686711-f116-4671-9451-e074b519fd78 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:07.514995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:07.532737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf686711-f116-4671-9451-e074b519fd78 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:07.566105 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3934b77-29c8-4850-9225-882b74a11f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:07.566710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934b77-29c8-4850-9225-882b74a11f0c None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:07.566994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934b77-29c8-4850-9225-882b74a11f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:07.567272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934b77-29c8-4850-9225-882b74a11f0c None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:07.568187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3934b77-29c8-4850-9225-882b74a11f0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:07.569659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3934b77-29c8-4850-9225-882b74a11f0c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:07.594688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 582/1177] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:32:07 2022] POST /identity/v3/users => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:32:09.517297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845440da-2651-4a97-abf8-8608fd88f304 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:09.517849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845440da-2651-4a97-abf8-8608fd88f304 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:09.517849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845440da-2651-4a97-abf8-8608fd88f304 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:09.519194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 596/1178] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:32:09.524951 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea824f31-b07c-4831-a043-017805d9f5e2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:09.525357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea824f31-b07c-4831-a043-017805d9f5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:09.525499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea824f31-b07c-4831-a043-017805d9f5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:09.543377 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ea824f31-b07c-4831-a043-017805d9f5e2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:32:09.544575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ea824f31-b07c-4831-a043-017805d9f5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:32:09.609315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 583/1179] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:32:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:32:09.876732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845440da-2651-4a97-abf8-8608fd88f304 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:09.960206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:09.977655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845440da-2651-4a97-abf8-8608fd88f304 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:10.011624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01351d60-40c4-4ac3-b659-266bbdfcb977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:10.012236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01351d60-40c4-4ac3-b659-266bbdfcb977 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:10.012540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01351d60-40c4-4ac3-b659-266bbdfcb977 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:10.012819 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01351d60-40c4-4ac3-b659-266bbdfcb977 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:10.022638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01351d60-40c4-4ac3-b659-266bbdfcb977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-01351d60-40c4-4ac3-b659-266bbdfcb977 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:32:10.028002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:32:10.029292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:32:10.030531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 597/1180] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:32:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:32:10.035092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea824f31-b07c-4831-a043-017805d9f5e2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:10.104124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:10.135119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea824f31-b07c-4831-a043-017805d9f5e2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:10.172289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d366254-4424-4de7-9fbd-4ffe1a4dadc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:10.172920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d366254-4424-4de7-9fbd-4ffe1a4dadc3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:10.173202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d366254-4424-4de7-9fbd-4ffe1a4dadc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:10.173472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d366254-4424-4de7-9fbd-4ffe1a4dadc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:10.174128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d366254-4424-4de7-9fbd-4ffe1a4dadc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:32:10.174501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d366254-4424-4de7-9fbd-4ffe1a4dadc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:10.175860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d366254-4424-4de7-9fbd-4ffe1a4dadc3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:10.184706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 584/1181] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:32:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:10.190492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01351d60-40c4-4ac3-b659-266bbdfcb977 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:10.288606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01351d60-40c4-4ac3-b659-266bbdfcb977 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:10.322435 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce64197-34dd-4387-b104-68b6b0b16c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:10.323042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce64197-34dd-4387-b104-68b6b0b16c97 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:10.323322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce64197-34dd-4387-b104-68b6b0b16c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:10.323594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce64197-34dd-4387-b104-68b6b0b16c97 None admin] PATH_INFO: `/v3/users/a1bf66b755de41b699403a3c303a8f80` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:10.336139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce64197-34dd-4387-b104-68b6b0b16c97 None admin] RBAC: Authorizing `identity:get_user(user_id=a1bf66b755de41b699403a3c303a8f80)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:10.337758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce64197-34dd-4387-b104-68b6b0b16c97 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:10.339324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 598/1182] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:32:10 2022] GET /identity/v3/users/a1bf66b755de41b699403a3c303a8f80 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:10.345184 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d366254-4424-4de7-9fbd-4ffe1a4dadc3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:10.443320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d366254-4424-4de7-9fbd-4ffe1a4dadc3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:10.472249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:10.478403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4c095-9bc9-4e2e-ae2e-91aa47b5e46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:10.479002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4c095-9bc9-4e2e-ae2e-91aa47b5e46a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:10.479284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4c095-9bc9-4e2e-ae2e-91aa47b5e46a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:10.479560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4c095-9bc9-4e2e-ae2e-91aa47b5e46a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:10.544565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4c095-9bc9-4e2e-ae2e-91aa47b5e46a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:10.546000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4c095-9bc9-4e2e-ae2e-91aa47b5e46a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:10.640562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 585/1183] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:32:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:32:10.646488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce64197-34dd-4387-b104-68b6b0b16c97 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:10.651332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:10.745220 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce64197-34dd-4387-b104-68b6b0b16c97 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:10.778204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a840b2-43fa-45d4-b87f-b05f5aca5631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:10.778815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a840b2-43fa-45d4-b87f-b05f5aca5631 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:10.779092 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a840b2-43fa-45d4-b87f-b05f5aca5631 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:10.779359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a840b2-43fa-45d4-b87f-b05f5aca5631 None admin] PATH_INFO: `/v3/projects/739d8a0a9210436ab06e5768c8028401` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:10.780445 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a840b2-43fa-45d4-b87f-b05f5aca5631 None admin] RBAC: Authorizing `identity:get_project(project_id=739d8a0a9210436ab06e5768c8028401)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:10.781887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a840b2-43fa-45d4-b87f-b05f5aca5631 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:10.783482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 599/1184] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:32:10 2022] GET /identity/v3/projects/739d8a0a9210436ab06e5768c8028401 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:10.789793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4c095-9bc9-4e2e-ae2e-91aa47b5e46a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:10.819156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:10.886153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4c095-9bc9-4e2e-ae2e-91aa47b5e46a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:10.919878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5889b22-6135-490e-901e-d4d4748b9f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:10.920469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889b22-6135-490e-901e-d4d4748b9f8a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:10.920784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889b22-6135-490e-901e-d4d4748b9f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:10.921057 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889b22-6135-490e-901e-d4d4748b9f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:10.921774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5889b22-6135-490e-901e-d4d4748b9f8a None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=739d8a0a9210436ab06e5768c8028401, user.id=a1bf66b755de41b699403a3c303a8f80) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:32:10.922083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5889b22-6135-490e-901e-d4d4748b9f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:10.923517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5889b22-6135-490e-901e-d4d4748b9f8a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:10.930231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 586/1185] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:32:10 2022] GET /identity/v3/role_assignments?user.id=a1bf66b755de41b699403a3c303a8f80&scope.project.id=739d8a0a9210436ab06e5768c8028401&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:12.888561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d5860-c1ad-42a5-ad4f-b75cdb08162e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:12.889093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d5860-c1ad-42a5-ad4f-b75cdb08162e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:12.889093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d5860-c1ad-42a5-ad4f-b75cdb08162e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:12.890418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 600/1186] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:32:12.895430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff547e-23a5-4a0b-a568-07fcda603e0d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:12.895850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff547e-23a5-4a0b-a568-07fcda603e0d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:12.895985 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff547e-23a5-4a0b-a568-07fcda603e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:12.915089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-48ff547e-23a5-4a0b-a568-07fcda603e0d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:32:12.916283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-48ff547e-23a5-4a0b-a568-07fcda603e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:32:12.988011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 587/1187] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:32:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:32:13.255897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d5860-c1ad-42a5-ad4f-b75cdb08162e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:13.335329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:13.365500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d5860-c1ad-42a5-ad4f-b75cdb08162e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:13.398632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426e7e4-3bbd-4201-97b1-cc7ca9ca0fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:13.399244 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e7e4-3bbd-4201-97b1-cc7ca9ca0fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:13.399521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e7e4-3bbd-4201-97b1-cc7ca9ca0fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:13.399791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e7e4-3bbd-4201-97b1-cc7ca9ca0fd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:13.409533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426e7e4-3bbd-4201-97b1-cc7ca9ca0fd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2426e7e4-3bbd-4201-97b1-cc7ca9ca0fd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:32:13.414393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:32:13.415567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:32:13.416483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 601/1188] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:32:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:32:13.421588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff547e-23a5-4a0b-a568-07fcda603e0d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:13.448352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:13.517979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff547e-23a5-4a0b-a568-07fcda603e0d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:13.549957 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45941cb-c902-4b25-a919-8ecd4fc1e5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:13.550553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45941cb-c902-4b25-a919-8ecd4fc1e5e5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:13.550832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45941cb-c902-4b25-a919-8ecd4fc1e5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:13.551104 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45941cb-c902-4b25-a919-8ecd4fc1e5e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:13.551766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45941cb-c902-4b25-a919-8ecd4fc1e5e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:32:13.552058 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45941cb-c902-4b25-a919-8ecd4fc1e5e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:13.553453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45941cb-c902-4b25-a919-8ecd4fc1e5e5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:13.561763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 588/1189] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:32:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:13.567924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426e7e4-3bbd-4201-97b1-cc7ca9ca0fd8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:13.679073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426e7e4-3bbd-4201-97b1-cc7ca9ca0fd8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:13.708865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:13.714131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961f730-2fc7-4e77-9570-703aa013f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:13.714750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961f730-2fc7-4e77-9570-703aa013f5bb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:13.715030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961f730-2fc7-4e77-9570-703aa013f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:13.715302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961f730-2fc7-4e77-9570-703aa013f5bb None admin] PATH_INFO: `/v3/users/a1bf66b755de41b699403a3c303a8f80` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:13.716401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961f730-2fc7-4e77-9570-703aa013f5bb None admin] RBAC: Authorizing `identity:get_user(user_id=a1bf66b755de41b699403a3c303a8f80)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:13.717846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961f730-2fc7-4e77-9570-703aa013f5bb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:13.719409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 602/1190] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:32:13 2022] GET /identity/v3/users/a1bf66b755de41b699403a3c303a8f80 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:13.725330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45941cb-c902-4b25-a919-8ecd4fc1e5e5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:13.812443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:13.824820 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45941cb-c902-4b25-a919-8ecd4fc1e5e5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:13.857960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b175-e3dc-4a0e-a0a3-4149770f1133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:13.858575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b175-e3dc-4a0e-a0a3-4149770f1133 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:13.858858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b175-e3dc-4a0e-a0a3-4149770f1133 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:13.859131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b175-e3dc-4a0e-a0a3-4149770f1133 None admin] PATH_INFO: `/v3/projects/739d8a0a9210436ab06e5768c8028401` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:13.860234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005b175-e3dc-4a0e-a0a3-4149770f1133 None admin] RBAC: Authorizing `identity:get_project(project_id=739d8a0a9210436ab06e5768c8028401)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:13.861700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005b175-e3dc-4a0e-a0a3-4149770f1133 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:13.863310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 589/1191] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:32:13 2022] GET /identity/v3/projects/739d8a0a9210436ab06e5768c8028401 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:13.870267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961f730-2fc7-4e77-9570-703aa013f5bb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:13.966787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961f730-2fc7-4e77-9570-703aa013f5bb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:13.999886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926aedb-a125-4289-80f0-66fe50585fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:14.000551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926aedb-a125-4289-80f0-66fe50585fbc None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:14.000837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926aedb-a125-4289-80f0-66fe50585fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:14.001112 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926aedb-a125-4289-80f0-66fe50585fbc None admin] PATH_INFO: `/v3/projects/739d8a0a9210436ab06e5768c8028401/users/a1bf66b755de41b699403a3c303a8f80/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:14.002965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926aedb-a125-4289-80f0-66fe50585fbc None admin] RBAC: Authorizing `identity:create_grant(project_id=739d8a0a9210436ab06e5768c8028401, user_id=a1bf66b755de41b699403a3c303a8f80, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:14.004384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926aedb-a125-4289-80f0-66fe50585fbc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:14.017017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 603/1192] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:32:13 2022] PUT /identity/v3/projects/739d8a0a9210436ab06e5768c8028401/users/a1bf66b755de41b699403a3c303a8f80/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:32:15.893191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957b2ad-8c75-4da0-8abe-ea5039f92470 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:15.893729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957b2ad-8c75-4da0-8abe-ea5039f92470 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:15.893729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957b2ad-8c75-4da0-8abe-ea5039f92470 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:15.895090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 590/1193] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:32:15.900214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc133d7-5d9c-4fb8-b477-fc05c5494dde None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:15.900685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc133d7-5d9c-4fb8-b477-fc05c5494dde None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:15.900788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc133d7-5d9c-4fb8-b477-fc05c5494dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:15.918689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9bc133d7-5d9c-4fb8-b477-fc05c5494dde None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:32:15.919857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9bc133d7-5d9c-4fb8-b477-fc05c5494dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:32:15.978576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:15.990067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 604/1194] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:32:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 11:32:16.259434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957b2ad-8c75-4da0-8abe-ea5039f92470 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:16.358925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957b2ad-8c75-4da0-8abe-ea5039f92470 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:16.392218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de017e82-efe8-4593-8a79-81ffd2556d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:16.392857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de017e82-efe8-4593-8a79-81ffd2556d15 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:16.393140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de017e82-efe8-4593-8a79-81ffd2556d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:16.393413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de017e82-efe8-4593-8a79-81ffd2556d15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:16.403248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de017e82-efe8-4593-8a79-81ffd2556d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-de017e82-efe8-4593-8a79-81ffd2556d15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:32:16.408025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:32:16.409136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:32:16.410140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 591/1195] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:32:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:32:16.415061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc133d7-5d9c-4fb8-b477-fc05c5494dde None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:16.511222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc133d7-5d9c-4fb8-b477-fc05c5494dde None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:16.542900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59a1d7-2b7a-4611-82ed-606531b039ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:16.543488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59a1d7-2b7a-4611-82ed-606531b039ea None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:16.543766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59a1d7-2b7a-4611-82ed-606531b039ea None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:16.544036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59a1d7-2b7a-4611-82ed-606531b039ea None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:16.544710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59a1d7-2b7a-4611-82ed-606531b039ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:32:16.545011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59a1d7-2b7a-4611-82ed-606531b039ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:16.546352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59a1d7-2b7a-4611-82ed-606531b039ea None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:16.554119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 605/1196] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:32:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:16.560473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de017e82-efe8-4593-8a79-81ffd2556d15 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:16.578979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:16.657364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de017e82-efe8-4593-8a79-81ffd2556d15 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:16.690588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf408e2-d4d4-48c4-a203-4799bdbd8062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:16.691217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf408e2-d4d4-48c4-a203-4799bdbd8062 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:16.691500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf408e2-d4d4-48c4-a203-4799bdbd8062 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:16.691774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf408e2-d4d4-48c4-a203-4799bdbd8062 None admin] PATH_INFO: `/v3/users/a1bf66b755de41b699403a3c303a8f80` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:16.693184 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf408e2-d4d4-48c4-a203-4799bdbd8062 None admin] RBAC: Authorizing `identity:get_user(user_id=a1bf66b755de41b699403a3c303a8f80)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:16.694607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf408e2-d4d4-48c4-a203-4799bdbd8062 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:16.696810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 592/1197] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Aug 31 11:32:16 2022] GET /identity/v3/users/a1bf66b755de41b699403a3c303a8f80 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:16.702647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59a1d7-2b7a-4611-82ed-606531b039ea None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:16.754821 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:16.800019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59a1d7-2b7a-4611-82ed-606531b039ea None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:16.832148 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8787e5-9c10-46ba-b0d3-3d776419c085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:16.832757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8787e5-9c10-46ba-b0d3-3d776419c085 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:16.833032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8787e5-9c10-46ba-b0d3-3d776419c085 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:16.833296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8787e5-9c10-46ba-b0d3-3d776419c085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:16.897267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8787e5-9c10-46ba-b0d3-3d776419c085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:16.898732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8787e5-9c10-46ba-b0d3-3d776419c085 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:16.948540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:16.994187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 606/1198] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:32:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:32:17.000146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf408e2-d4d4-48c4-a203-4799bdbd8062 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:17.081101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:17.096877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf408e2-d4d4-48c4-a203-4799bdbd8062 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:17.129575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4e4a1-c44d-4a12-88f6-0bc8669481ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:17.130187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4e4a1-c44d-4a12-88f6-0bc8669481ae None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:17.130466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4e4a1-c44d-4a12-88f6-0bc8669481ae None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:17.130741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4e4a1-c44d-4a12-88f6-0bc8669481ae None admin] PATH_INFO: `/v3/projects/739d8a0a9210436ab06e5768c8028401` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:17.131844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4e4a1-c44d-4a12-88f6-0bc8669481ae None admin] RBAC: Authorizing `identity:get_project(project_id=739d8a0a9210436ab06e5768c8028401)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:17.133413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4e4a1-c44d-4a12-88f6-0bc8669481ae None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:17.135027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 593/1199] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Aug 31 11:32:16 2022] GET /identity/v3/projects/739d8a0a9210436ab06e5768c8028401 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:17.141161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8787e5-9c10-46ba-b0d3-3d776419c085 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:32:17.239011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8787e5-9c10-46ba-b0d3-3d776419c085 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:32:17.271677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487ea4f-81e6-468c-b31f-2cf6d7055dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:32:17.272265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487ea4f-81e6-468c-b31f-2cf6d7055dbf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:32:17.272581 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487ea4f-81e6-468c-b31f-2cf6d7055dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:32:17.272857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487ea4f-81e6-468c-b31f-2cf6d7055dbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:32:17.274169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487ea4f-81e6-468c-b31f-2cf6d7055dbf None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=739d8a0a9210436ab06e5768c8028401, user.id=a1bf66b755de41b699403a3c303a8f80) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:32:17.274491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487ea4f-81e6-468c-b31f-2cf6d7055dbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:32:17.275897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487ea4f-81e6-468c-b31f-2cf6d7055dbf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:32:17.281515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 607/1200] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:32:17 2022] GET /identity/v3/role_assignments?user.id=a1bf66b755de41b699403a3c303a8f80&scope.project.id=739d8a0a9210436ab06e5768c8028401&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:32:27.125552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=24 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:32:27.278977 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:12.890457 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63808c70-17fb-4e22-976e-afc01ca8941c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:12.891167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63808c70-17fb-4e22-976e-afc01ca8941c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:12.891167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63808c70-17fb-4e22-976e-afc01ca8941c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:12.892391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 594/1201] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:33:12.899746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3292361-5cc9-4bcd-a91d-c9449f1d4f99 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:12.900699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3292361-5cc9-4bcd-a91d-c9449f1d4f99 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:12.901011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3292361-5cc9-4bcd-a91d-c9449f1d4f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:12.926583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f3292361-5cc9-4bcd-a91d-c9449f1d4f99 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:33:12.927888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f3292361-5cc9-4bcd-a91d-c9449f1d4f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:13.033512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 608/1202] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:33:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:33:13.368263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63808c70-17fb-4e22-976e-afc01ca8941c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:13.531752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63808c70-17fb-4e22-976e-afc01ca8941c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:13.580042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba090d-736e-4691-8e9a-31ea28bb4df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:13.580740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba090d-736e-4691-8e9a-31ea28bb4df8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:13.581005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba090d-736e-4691-8e9a-31ea28bb4df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:13.581263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba090d-736e-4691-8e9a-31ea28bb4df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:13.598779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba090d-736e-4691-8e9a-31ea28bb4df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:13.601488 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba090d-736e-4691-8e9a-31ea28bb4df8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-ceba090d-736e-4691-8e9a-31ea28bb4df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:13.628935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:13.630048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:13.631319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:13.631319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:13.631319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:13.631319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 595/1203] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:33:13.639885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3292361-5cc9-4bcd-a91d-c9449f1d4f99 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:13.803644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3292361-5cc9-4bcd-a91d-c9449f1d4f99 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:13.853396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:13.863764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f0ef0-ac0f-495b-b4f8-1c488b156772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:13.864562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0ef0-ac0f-495b-b4f8-1c488b156772 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:13.864882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0ef0-ac0f-495b-b4f8-1c488b156772 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:13.865167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0ef0-ac0f-495b-b4f8-1c488b156772 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:13.865879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f0ef0-ac0f-495b-b4f8-1c488b156772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:13.866190 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f0ef0-ac0f-495b-b4f8-1c488b156772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:13.867757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f0ef0-ac0f-495b-b4f8-1c488b156772 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:13.883367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 609/1204] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:13.889739 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba090d-736e-4691-8e9a-31ea28bb4df8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:13.957058 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:14.015838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba090d-736e-4691-8e9a-31ea28bb4df8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:14.064579 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a540bc2-014e-480a-89b5-e090d6c37b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:14.065221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a540bc2-014e-480a-89b5-e090d6c37b41 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:14.065506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a540bc2-014e-480a-89b5-e090d6c37b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:14.065779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a540bc2-014e-480a-89b5-e090d6c37b41 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:14.066485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a540bc2-014e-480a-89b5-e090d6c37b41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:14.067987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a540bc2-014e-480a-89b5-e090d6c37b41 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:14.095399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 596/1205] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Aug 31 11:33:13 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:33:16.288870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dff328-d132-416c-ac34-64c3b5e8fad8 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:16.289422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dff328-d132-416c-ac34-64c3b5e8fad8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:16.289422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dff328-d132-416c-ac34-64c3b5e8fad8 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:16.290929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 610/1206] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:33:16.296664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92e00-4222-454a-9429-051ed3d2a844 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:16.297155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92e00-4222-454a-9429-051ed3d2a844 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:16.297261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92e00-4222-454a-9429-051ed3d2a844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:16.319076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b2f92e00-4222-454a-9429-051ed3d2a844 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:33:16.320441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b2f92e00-4222-454a-9429-051ed3d2a844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:16.399180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 597/1207] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:33:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:33:16.688395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dff328-d132-416c-ac34-64c3b5e8fad8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:16.810570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dff328-d132-416c-ac34-64c3b5e8fad8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:16.850228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90216b6b-2189-4cbf-8609-3595ecfd4195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:16.851517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90216b6b-2189-4cbf-8609-3595ecfd4195 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:16.851784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90216b6b-2189-4cbf-8609-3595ecfd4195 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:16.852084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90216b6b-2189-4cbf-8609-3595ecfd4195 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:16.863128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90216b6b-2189-4cbf-8609-3595ecfd4195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-90216b6b-2189-4cbf-8609-3595ecfd4195 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:33:16.870101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:16.871134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:16.873173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 611/1208] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:33:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:16.877510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f92e00-4222-454a-9429-051ed3d2a844 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:16.922282 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:16.993508 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f92e00-4222-454a-9429-051ed3d2a844 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:17.040094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6f298-6d1e-424d-ba9e-0e35a395d0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:17.041335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6f298-6d1e-424d-ba9e-0e35a395d0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:17.041692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6f298-6d1e-424d-ba9e-0e35a395d0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:17.042041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6f298-6d1e-424d-ba9e-0e35a395d0f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:17.042950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6f298-6d1e-424d-ba9e-0e35a395d0f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:17.043299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6f298-6d1e-424d-ba9e-0e35a395d0f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:17.044983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6f298-6d1e-424d-ba9e-0e35a395d0f5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:17.061360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 598/1209] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:33:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:17.062776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90216b6b-2189-4cbf-8609-3595ecfd4195 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:17.130710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:17.199385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90216b6b-2189-4cbf-8609-3595ecfd4195 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:17.247938 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cf3ad-d9b3-4a00-87d4-c773c205e279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:17.248505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cf3ad-d9b3-4a00-87d4-c773c205e279 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:17.248798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cf3ad-d9b3-4a00-87d4-c773c205e279 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:17.249076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cf3ad-d9b3-4a00-87d4-c773c205e279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:17.260953 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cf3ad-d9b3-4a00-87d4-c773c205e279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:17.262443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cf3ad-d9b3-4a00-87d4-c773c205e279 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-524cf3ad-d9b3-4a00-87d4-c773c205e279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:17.280084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:17.281371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:17.282263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:17.282263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:17.282263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:17.282263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 612/1210] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:17.288254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6f298-6d1e-424d-ba9e-0e35a395d0f5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:17.410688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:17.414412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6f298-6d1e-424d-ba9e-0e35a395d0f5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:17.454340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499feb16-7217-4ac6-8d56-e55e4c7d8236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:17.454986 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499feb16-7217-4ac6-8d56-e55e4c7d8236 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:17.455265 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499feb16-7217-4ac6-8d56-e55e4c7d8236 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:17.455642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499feb16-7217-4ac6-8d56-e55e4c7d8236 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:17.456274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499feb16-7217-4ac6-8d56-e55e4c7d8236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:17.456605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499feb16-7217-4ac6-8d56-e55e4c7d8236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:17.458050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499feb16-7217-4ac6-8d56-e55e4c7d8236 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:17.471608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 599/1211] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:17.478948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cf3ad-d9b3-4a00-87d4-c773c205e279 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:17.595149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cf3ad-d9b3-4a00-87d4-c773c205e279 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:17.596919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:17.636716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424992d3-ec2a-40fd-8b04-ad0b498a0064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:17.637824 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424992d3-ec2a-40fd-8b04-ad0b498a0064 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:17.638149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424992d3-ec2a-40fd-8b04-ad0b498a0064 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:17.638488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424992d3-ec2a-40fd-8b04-ad0b498a0064 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:17.646841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424992d3-ec2a-40fd-8b04-ad0b498a0064 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:17.649463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424992d3-ec2a-40fd-8b04-ad0b498a0064 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-424992d3-ec2a-40fd-8b04-ad0b498a0064 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:33:17.662417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:17.663665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:17.670259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:17.670259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:17.670259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:33:17.670259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:33:17.670259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:33:17.670259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:33:17.670259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:33:17.670259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:33:17.670259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:17.670259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:17.670259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 613/1212] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:17.670892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499feb16-7217-4ac6-8d56-e55e4c7d8236 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:17.791060 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499feb16-7217-4ac6-8d56-e55e4c7d8236 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:17.829733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8163a3-ed65-4f2f-90ca-873a3849de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:17.830348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8163a3-ed65-4f2f-90ca-873a3849de5c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:17.830629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8163a3-ed65-4f2f-90ca-873a3849de5c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:17.832586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8163a3-ed65-4f2f-90ca-873a3849de5c None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:17.832586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8163a3-ed65-4f2f-90ca-873a3849de5c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:17.832586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8163a3-ed65-4f2f-90ca-873a3849de5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:17.833698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8163a3-ed65-4f2f-90ca-873a3849de5c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:17.854639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 600/1213] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Aug 31 11:33:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:17.860849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424992d3-ec2a-40fd-8b04-ad0b498a0064 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:17.985479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424992d3-ec2a-40fd-8b04-ad0b498a0064 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:18.030472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648bf68-4a6e-4533-8925-9f33c2ad93db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:18.031186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648bf68-4a6e-4533-8925-9f33c2ad93db None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:18.031510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648bf68-4a6e-4533-8925-9f33c2ad93db None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:18.031796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648bf68-4a6e-4533-8925-9f33c2ad93db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:18.070023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:18.116776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b648bf68-4a6e-4533-8925-9f33c2ad93db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:18.118139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b648bf68-4a6e-4533-8925-9f33c2ad93db None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:18.251324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 614/1214] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:33:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:33:18.257387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8163a3-ed65-4f2f-90ca-873a3849de5c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:18.281791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:18.379160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8163a3-ed65-4f2f-90ca-873a3849de5c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:18.429894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf52e70-df2a-4390-82ea-aeb88d946bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:18.430630 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf52e70-df2a-4390-82ea-aeb88d946bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:18.430885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf52e70-df2a-4390-82ea-aeb88d946bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:18.431139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf52e70-df2a-4390-82ea-aeb88d946bf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:18.444982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf52e70-df2a-4390-82ea-aeb88d946bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:18.446479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf52e70-df2a-4390-82ea-aeb88d946bf1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-adf52e70-df2a-4390-82ea-aeb88d946bf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:18.463861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:18.464947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:18.465837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:18.465837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:18.465837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:18.465837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 601/1215] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:18.470743 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648bf68-4a6e-4533-8925-9f33c2ad93db None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:18.543866 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:18.592647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648bf68-4a6e-4533-8925-9f33c2ad93db None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:18.630522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286bc2a-8a93-4615-95ea-7d2bf4487718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:18.631126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286bc2a-8a93-4615-95ea-7d2bf4487718 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:18.631405 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286bc2a-8a93-4615-95ea-7d2bf4487718 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:18.631712 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286bc2a-8a93-4615-95ea-7d2bf4487718 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:18.632768 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286bc2a-8a93-4615-95ea-7d2bf4487718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:18.633259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286bc2a-8a93-4615-95ea-7d2bf4487718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:18.634604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286bc2a-8a93-4615-95ea-7d2bf4487718 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:18.647262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 615/1216] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:18.653275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf52e70-df2a-4390-82ea-aeb88d946bf1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:18.742377 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:18.778153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf52e70-df2a-4390-82ea-aeb88d946bf1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:18.816072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1000d1-92c7-45b2-b088-3f8a09844406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:18.816789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1000d1-92c7-45b2-b088-3f8a09844406 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:18.817085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1000d1-92c7-45b2-b088-3f8a09844406 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:18.817353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1000d1-92c7-45b2-b088-3f8a09844406 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:18.826614 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1000d1-92c7-45b2-b088-3f8a09844406 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:18.828331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1000d1-92c7-45b2-b088-3f8a09844406 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-1c1000d1-92c7-45b2-b088-3f8a09844406 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:33:18.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:18.843600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:18.844556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:33:18.844556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:33:18.844556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:33:18.844556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:33:18.844556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:33:18.844556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:33:18.844556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:18.844556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:18.844556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 602/1217] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:33:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:18.849853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286bc2a-8a93-4615-95ea-7d2bf4487718 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:18.976055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286bc2a-8a93-4615-95ea-7d2bf4487718 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:19.004305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:19.020040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb142e-78bf-4191-8cba-94a32fd552cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:19.020760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb142e-78bf-4191-8cba-94a32fd552cc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:19.021090 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb142e-78bf-4191-8cba-94a32fd552cc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:19.021370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb142e-78bf-4191-8cba-94a32fd552cc None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:19.022078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb142e-78bf-4191-8cba-94a32fd552cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:19.022444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb142e-78bf-4191-8cba-94a32fd552cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:19.024632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb142e-78bf-4191-8cba-94a32fd552cc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:19.037338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 616/1218] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:33:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:19.043716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1000d1-92c7-45b2-b088-3f8a09844406 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:19.158363 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1000d1-92c7-45b2-b088-3f8a09844406 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:19.191882 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:19.198660 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1ed83-fe83-45eb-b63d-fc6806cdfdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:19.199265 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1ed83-fe83-45eb-b63d-fc6806cdfdae None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:19.199559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1ed83-fe83-45eb-b63d-fc6806cdfdae None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:19.199814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1ed83-fe83-45eb-b63d-fc6806cdfdae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:19.200568 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1ed83-fe83-45eb-b63d-fc6806cdfdae None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=f9114a71a3a247188b263d87b6cb12c6) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:19.200878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1ed83-fe83-45eb-b63d-fc6806cdfdae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:19.202308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1ed83-fe83-45eb-b63d-fc6806cdfdae None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:19.207761 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 603/1219] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:33:19 2022] GET /identity/v3/role_assignments?user.id=f9114a71a3a247188b263d87b6cb12c6&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:21.256132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508731e-ee39-4f32-862e-7895690a7888 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:21.256697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508731e-ee39-4f32-862e-7895690a7888 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:21.256697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508731e-ee39-4f32-862e-7895690a7888 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:21.258694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 617/1220] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:33:21.263958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fc623-9496-4e1e-8efc-fa553ba689ed None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:21.264417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fc623-9496-4e1e-8efc-fa553ba689ed None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:21.264551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fc623-9496-4e1e-8efc-fa553ba689ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:21.282845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f66fc623-9496-4e1e-8efc-fa553ba689ed None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:33:21.284037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f66fc623-9496-4e1e-8efc-fa553ba689ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:21.357953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 604/1221] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:33:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:33:21.645184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508731e-ee39-4f32-862e-7895690a7888 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:21.754769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508731e-ee39-4f32-862e-7895690a7888 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:21.789890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5d1b1-0062-401c-ba21-f2f8ac14222a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:21.790516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5d1b1-0062-401c-ba21-f2f8ac14222a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:21.790796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5d1b1-0062-401c-ba21-f2f8ac14222a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:21.791071 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5d1b1-0062-401c-ba21-f2f8ac14222a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:21.801920 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5d1b1-0062-401c-ba21-f2f8ac14222a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-23f5d1b1-0062-401c-ba21-f2f8ac14222a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:33:21.807304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:21.812606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:21.813780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 618/1222] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:33:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:21.814965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fc623-9496-4e1e-8efc-fa553ba689ed None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:21.919064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fc623-9496-4e1e-8efc-fa553ba689ed None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:21.942895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:21.954941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc0a71-0416-4630-b9f3-a7b4388338d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:21.955581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc0a71-0416-4630-b9f3-a7b4388338d9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:21.955859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc0a71-0416-4630-b9f3-a7b4388338d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:21.956156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc0a71-0416-4630-b9f3-a7b4388338d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:21.956898 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc0a71-0416-4630-b9f3-a7b4388338d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:21.957200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc0a71-0416-4630-b9f3-a7b4388338d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:21.958675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc0a71-0416-4630-b9f3-a7b4388338d9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:21.967406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 605/1223] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:33:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:21.974104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5d1b1-0062-401c-ba21-f2f8ac14222a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:22.019946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:22.079702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5d1b1-0062-401c-ba21-f2f8ac14222a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:22.114708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d2877-e2fe-4f48-9d82-561d764a5525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:22.115324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d2877-e2fe-4f48-9d82-561d764a5525 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:22.115597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d2877-e2fe-4f48-9d82-561d764a5525 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:22.115861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d2877-e2fe-4f48-9d82-561d764a5525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:22.126929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d2877-e2fe-4f48-9d82-561d764a5525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:22.128475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d2877-e2fe-4f48-9d82-561d764a5525 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-470d2877-e2fe-4f48-9d82-561d764a5525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:22.144434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:22.145473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:22.146412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:22.146412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:22.146412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:22.146412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 619/1224] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:22.151654 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc0a71-0416-4630-b9f3-a7b4388338d9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:22.266580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc0a71-0416-4630-b9f3-a7b4388338d9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:22.302961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d4da6-e585-42b6-b3ed-17393c00095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:22.303571 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d4da6-e585-42b6-b3ed-17393c00095b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:22.304403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d4da6-e585-42b6-b3ed-17393c00095b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:22.304764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d4da6-e585-42b6-b3ed-17393c00095b None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:22.305375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d4da6-e585-42b6-b3ed-17393c00095b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:22.305680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d4da6-e585-42b6-b3ed-17393c00095b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:22.307069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d4da6-e585-42b6-b3ed-17393c00095b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:22.319809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 606/1225] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:22.329745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d2877-e2fe-4f48-9d82-561d764a5525 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:22.436228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:22.445060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d2877-e2fe-4f48-9d82-561d764a5525 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:22.485304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dbae9-44cf-4075-91a6-74a405b88cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:22.485997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dbae9-44cf-4075-91a6-74a405b88cef None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:22.486378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dbae9-44cf-4075-91a6-74a405b88cef None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:22.486777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dbae9-44cf-4075-91a6-74a405b88cef None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:22.495045 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7dbae9-44cf-4075-91a6-74a405b88cef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:22.496724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7dbae9-44cf-4075-91a6-74a405b88cef None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-fe7dbae9-44cf-4075-91a6-74a405b88cef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:33:22.508675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:22.509789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:22.511103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:22.511103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:22.511103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:33:22.511103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:33:22.511103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:33:22.511103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:33:22.511103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:33:22.511103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:33:22.511103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:22.511103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:22.511103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 620/1226] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:22.515911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d4da6-e585-42b6-b3ed-17393c00095b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:22.570600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:22.631059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d4da6-e585-42b6-b3ed-17393c00095b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:22.669092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758d145-1118-4ab4-bdaa-edb820906005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:22.669702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d145-1118-4ab4-bdaa-edb820906005 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:22.669999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d145-1118-4ab4-bdaa-edb820906005 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:22.670256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d145-1118-4ab4-bdaa-edb820906005 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:22.670994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758d145-1118-4ab4-bdaa-edb820906005 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:22.671296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758d145-1118-4ab4-bdaa-edb820906005 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:22.672762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758d145-1118-4ab4-bdaa-edb820906005 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:22.692388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 607/1227] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Aug 31 11:33:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:33:22.700338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dbae9-44cf-4075-91a6-74a405b88cef None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:22.813221 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dbae9-44cf-4075-91a6-74a405b88cef None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:22.854514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ff0c5-6a52-400e-a028-e9fce88ba9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:22.855074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff0c5-6a52-400e-a028-e9fce88ba9df None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:22.855358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff0c5-6a52-400e-a028-e9fce88ba9df None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:22.855705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff0c5-6a52-400e-a028-e9fce88ba9df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:22.868744 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ff0c5-6a52-400e-a028-e9fce88ba9df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:22.870433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ff0c5-6a52-400e-a028-e9fce88ba9df None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-702ff0c5-6a52-400e-a028-e9fce88ba9df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:22.886585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:22.887928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:22.889340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:22.889340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:22.889340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:22.889340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 621/1228] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:22.894073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758d145-1118-4ab4-bdaa-edb820906005 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:22.998626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758d145-1118-4ab4-bdaa-edb820906005 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:23.001541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:23.036116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568fb2f-ed52-4cd9-8dc9-ee5627012d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:23.036748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568fb2f-ed52-4cd9-8dc9-ee5627012d54 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:23.037042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568fb2f-ed52-4cd9-8dc9-ee5627012d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:23.037299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568fb2f-ed52-4cd9-8dc9-ee5627012d54 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:23.038513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568fb2f-ed52-4cd9-8dc9-ee5627012d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:23.039019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568fb2f-ed52-4cd9-8dc9-ee5627012d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:23.040411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568fb2f-ed52-4cd9-8dc9-ee5627012d54 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:23.051546 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 608/1229] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:23.058168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ff0c5-6a52-400e-a028-e9fce88ba9df None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:23.069006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:23.170410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ff0c5-6a52-400e-a028-e9fce88ba9df None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:23.216184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08377df-1833-436f-9df0-a5b05fa284f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:23.216902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08377df-1833-436f-9df0-a5b05fa284f7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:23.217193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08377df-1833-436f-9df0-a5b05fa284f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:23.217548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08377df-1833-436f-9df0-a5b05fa284f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:23.225147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08377df-1833-436f-9df0-a5b05fa284f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:23.227468 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08377df-1833-436f-9df0-a5b05fa284f7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b08377df-1833-436f-9df0-a5b05fa284f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:33:23.239332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:23.240230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:23.241419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:33:23.241419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:33:23.241419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:33:23.241419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:33:23.241419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:33:23.241419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:33:23.241419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:23.241419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:23.241419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 622/1230] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:33:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:23.246581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568fb2f-ed52-4cd9-8dc9-ee5627012d54 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:23.362957 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568fb2f-ed52-4cd9-8dc9-ee5627012d54 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:23.401130 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae4128-a51e-49ae-980b-5e8730eada86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:23.401954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae4128-a51e-49ae-980b-5e8730eada86 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:23.402474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae4128-a51e-49ae-980b-5e8730eada86 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:23.402760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae4128-a51e-49ae-980b-5e8730eada86 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:23.403581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae4128-a51e-49ae-980b-5e8730eada86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:23.403890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae4128-a51e-49ae-980b-5e8730eada86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:23.405700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae4128-a51e-49ae-980b-5e8730eada86 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:23.421424 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 609/1231] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:33:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:33:23.427651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08377df-1833-436f-9df0-a5b05fa284f7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:23.500802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:23.541027 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08377df-1833-436f-9df0-a5b05fa284f7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:23.577754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0df01a-c74d-4b21-b070-5d7421b409a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:23.578422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df01a-c74d-4b21-b070-5d7421b409a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:23.578696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df01a-c74d-4b21-b070-5d7421b409a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:23.578962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df01a-c74d-4b21-b070-5d7421b409a8 None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/f9114a71a3a247188b263d87b6cb12c6/roles/babbae9ac71c4c15817721af5d270a80` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:23.593411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0df01a-c74d-4b21-b070-5d7421b409a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=f9114a71a3a247188b263d87b6cb12c6, role_id=babbae9ac71c4c15817721af5d270a80)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:23.594869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0df01a-c74d-4b21-b070-5d7421b409a8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:23.608677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 623/1232] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:33:23 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/f9114a71a3a247188b263d87b6cb12c6/roles/babbae9ac71c4c15817721af5d270a80 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:33:25.540626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97510672-290e-4926-8791-2565ffd51f94 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:25.541173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97510672-290e-4926-8791-2565ffd51f94 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:25.541173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97510672-290e-4926-8791-2565ffd51f94 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:25.542537 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 610/1233] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:33:25.547754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a0304-7bf8-42cb-9b9e-0bba6ba930e2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:25.548186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a0304-7bf8-42cb-9b9e-0bba6ba930e2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:25.548333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a0304-7bf8-42cb-9b9e-0bba6ba930e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:25.566717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9e6a0304-7bf8-42cb-9b9e-0bba6ba930e2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:33:25.567899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9e6a0304-7bf8-42cb-9b9e-0bba6ba930e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:25.640175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 624/1234] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:33:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:33:25.922659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97510672-290e-4926-8791-2565ffd51f94 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:25.951744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:26.039311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97510672-290e-4926-8791-2565ffd51f94 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:26.078736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5ea19-5da2-487e-9b2d-f64f027fa555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:26.079408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5ea19-5da2-487e-9b2d-f64f027fa555 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:26.079686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5ea19-5da2-487e-9b2d-f64f027fa555 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:26.079958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5ea19-5da2-487e-9b2d-f64f027fa555 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:26.091236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5ea19-5da2-487e-9b2d-f64f027fa555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-98c5ea19-5da2-487e-9b2d-f64f027fa555 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:33:26.096464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:26.097556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:26.098589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 611/1235] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Aug 31 11:33:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:26.104385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a0304-7bf8-42cb-9b9e-0bba6ba930e2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:26.153656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:26.219648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a0304-7bf8-42cb-9b9e-0bba6ba930e2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:26.258316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab715c1-a349-4ebd-ae0f-baf565021250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:26.258921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab715c1-a349-4ebd-ae0f-baf565021250 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:26.259201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab715c1-a349-4ebd-ae0f-baf565021250 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:26.259485 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab715c1-a349-4ebd-ae0f-baf565021250 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:26.260234 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab715c1-a349-4ebd-ae0f-baf565021250 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:26.260545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab715c1-a349-4ebd-ae0f-baf565021250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:26.261979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab715c1-a349-4ebd-ae0f-baf565021250 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:26.271011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 625/1236] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Aug 31 11:33:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:26.278028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5ea19-5da2-487e-9b2d-f64f027fa555 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:26.379306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:26.399031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5ea19-5da2-487e-9b2d-f64f027fa555 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:26.438295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cc1c2-3662-4a16-a9c3-289eb72e2665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:26.438977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc1c2-3662-4a16-a9c3-289eb72e2665 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:26.439256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc1c2-3662-4a16-a9c3-289eb72e2665 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:26.439535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc1c2-3662-4a16-a9c3-289eb72e2665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:26.451657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cc1c2-3662-4a16-a9c3-289eb72e2665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:26.453205 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cc1c2-3662-4a16-a9c3-289eb72e2665 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-a94cc1c2-3662-4a16-a9c3-289eb72e2665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:26.468917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:26.470216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:26.471333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:26.471333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:26.471333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:26.471333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 612/1237] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:26.477495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab715c1-a349-4ebd-ae0f-baf565021250 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:26.589223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab715c1-a349-4ebd-ae0f-baf565021250 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:26.599851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:26.627668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fdeb9-e8c2-44e0-9b8a-96b440dc55ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:26.628277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fdeb9-e8c2-44e0-9b8a-96b440dc55ff None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:26.628587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fdeb9-e8c2-44e0-9b8a-96b440dc55ff None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:26.628865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fdeb9-e8c2-44e0-9b8a-96b440dc55ff None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:26.629499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fdeb9-e8c2-44e0-9b8a-96b440dc55ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:26.629800 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fdeb9-e8c2-44e0-9b8a-96b440dc55ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:26.631198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fdeb9-e8c2-44e0-9b8a-96b440dc55ff None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:26.643982 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 626/1238] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:26.650115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cc1c2-3662-4a16-a9c3-289eb72e2665 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:26.770172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cc1c2-3662-4a16-a9c3-289eb72e2665 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:26.807695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58822e58-efbe-4d27-8ec8-73dae501a862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:26.808371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58822e58-efbe-4d27-8ec8-73dae501a862 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:26.808808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58822e58-efbe-4d27-8ec8-73dae501a862 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:26.809096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58822e58-efbe-4d27-8ec8-73dae501a862 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:26.812234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:26.817960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58822e58-efbe-4d27-8ec8-73dae501a862 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:26.820185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58822e58-efbe-4d27-8ec8-73dae501a862 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-58822e58-efbe-4d27-8ec8-73dae501a862 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:33:26.832962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:26.834188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:26.836027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:26.836027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:26.836027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:33:26.836027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:33:26.836027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:33:26.836027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:33:26.836027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:33:26.836027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:33:26.836027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:26.836027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:26.836027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 613/1239] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:26.840886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fdeb9-e8c2-44e0-9b8a-96b440dc55ff None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:26.958849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fdeb9-e8c2-44e0-9b8a-96b440dc55ff None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:26.996040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98ef3f-5220-4b1f-b616-e3ff0a865f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:26.996778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98ef3f-5220-4b1f-b616-e3ff0a865f33 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:26.997058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98ef3f-5220-4b1f-b616-e3ff0a865f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:26.997414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98ef3f-5220-4b1f-b616-e3ff0a865f33 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:26.998253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98ef3f-5220-4b1f-b616-e3ff0a865f33 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:26.998562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98ef3f-5220-4b1f-b616-e3ff0a865f33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:27.000149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98ef3f-5220-4b1f-b616-e3ff0a865f33 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:27.018049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 627/1240] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Aug 31 11:33:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:27.025189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58822e58-efbe-4d27-8ec8-73dae501a862 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:27.141203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58822e58-efbe-4d27-8ec8-73dae501a862 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:27.180156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cac1ef-c00f-4066-bdeb-c9b3b44e7de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:27.180819 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cac1ef-c00f-4066-bdeb-c9b3b44e7de6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:27.181140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cac1ef-c00f-4066-bdeb-c9b3b44e7de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:27.181422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cac1ef-c00f-4066-bdeb-c9b3b44e7de6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:27.252989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:27.260262 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cac1ef-c00f-4066-bdeb-c9b3b44e7de6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:27.262162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cac1ef-c00f-4066-bdeb-c9b3b44e7de6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:27.381436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 614/1241] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:33:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:33:27.389162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98ef3f-5220-4b1f-b616-e3ff0a865f33 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:27.420691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:27.503347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98ef3f-5220-4b1f-b616-e3ff0a865f33 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:27.544578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752f1c3-f206-44c6-831b-a1b7162fa61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:27.545346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752f1c3-f206-44c6-831b-a1b7162fa61b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:27.545552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752f1c3-f206-44c6-831b-a1b7162fa61b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:27.545842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752f1c3-f206-44c6-831b-a1b7162fa61b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:27.557905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752f1c3-f206-44c6-831b-a1b7162fa61b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:27.559556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752f1c3-f206-44c6-831b-a1b7162fa61b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0752f1c3-f206-44c6-831b-a1b7162fa61b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:27.575681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:27.576847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:27.577826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:27.577826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:27.577826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:27.577826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 628/1242] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:27.583514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cac1ef-c00f-4066-bdeb-c9b3b44e7de6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:27.687077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:27.695726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cac1ef-c00f-4066-bdeb-c9b3b44e7de6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:27.736098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb243fd-7165-4fa4-bd0b-3414e863d6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:27.736717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb243fd-7165-4fa4-bd0b-3414e863d6bc None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:27.737002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb243fd-7165-4fa4-bd0b-3414e863d6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:27.737277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb243fd-7165-4fa4-bd0b-3414e863d6bc None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:27.737937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb243fd-7165-4fa4-bd0b-3414e863d6bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:27.738223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb243fd-7165-4fa4-bd0b-3414e863d6bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:27.739894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb243fd-7165-4fa4-bd0b-3414e863d6bc None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:27.753253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 615/1243] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:33:27.760727 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752f1c3-f206-44c6-831b-a1b7162fa61b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:27.843366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:27.869467 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752f1c3-f206-44c6-831b-a1b7162fa61b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:27.906081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff8fdb-db30-4426-aa13-ca287c15b760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:27.906855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8fdb-db30-4426-aa13-ca287c15b760 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:27.907114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8fdb-db30-4426-aa13-ca287c15b760 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:27.907408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8fdb-db30-4426-aa13-ca287c15b760 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:27.916727 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff8fdb-db30-4426-aa13-ca287c15b760 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:27.918253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff8fdb-db30-4426-aa13-ca287c15b760 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-33ff8fdb-db30-4426-aa13-ca287c15b760 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:33:27.931534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:27.932832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:27.933905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:33:27.933905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:33:27.933905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:33:27.933905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:33:27.933905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:33:27.933905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:33:27.933905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:27.933905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:27.933905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 629/1244] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:33:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:27.939476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb243fd-7165-4fa4-bd0b-3414e863d6bc None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:28.046300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb243fd-7165-4fa4-bd0b-3414e863d6bc None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:28.083483 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c231f-4001-4d5e-9b36-b396a8778751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:28.084092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c231f-4001-4d5e-9b36-b396a8778751 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:28.084370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c231f-4001-4d5e-9b36-b396a8778751 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:28.084692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c231f-4001-4d5e-9b36-b396a8778751 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:28.086169 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c231f-4001-4d5e-9b36-b396a8778751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:28.086491 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c231f-4001-4d5e-9b36-b396a8778751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:28.088079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c231f-4001-4d5e-9b36-b396a8778751 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:28.100147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 616/1245] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:33:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:28.106990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff8fdb-db30-4426-aa13-ca287c15b760 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:28.226065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff8fdb-db30-4426-aa13-ca287c15b760 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:28.261387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a38726-443f-4455-9439-17de0acf37d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:28.261983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a38726-443f-4455-9439-17de0acf37d9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:28.262264 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a38726-443f-4455-9439-17de0acf37d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:28.262529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a38726-443f-4455-9439-17de0acf37d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:28.263265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a38726-443f-4455-9439-17de0acf37d9 None admin] RBAC: Adding query filter params (role.id=babbae9ac71c4c15817721af5d270a80, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=f9114a71a3a247188b263d87b6cb12c6) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:28.263569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a38726-443f-4455-9439-17de0acf37d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:28.265088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a38726-443f-4455-9439-17de0acf37d9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:28.271055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 630/1246] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:33:28 2022] GET /identity/v3/role_assignments?user.id=f9114a71a3a247188b263d87b6cb12c6&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=babbae9ac71c4c15817721af5d270a80 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:30.384460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b01a5-8440-4e1a-8ee0-ddabbb2497cb None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:30.385016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b01a5-8440-4e1a-8ee0-ddabbb2497cb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:30.385127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b01a5-8440-4e1a-8ee0-ddabbb2497cb None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:30.386467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 617/1247] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:33:30.392186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7994ab-c2ea-4d58-89f4-6fec469aeb70 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:30.392777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7994ab-c2ea-4d58-89f4-6fec469aeb70 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:30.392777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7994ab-c2ea-4d58-89f4-6fec469aeb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:30.400562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:30.415848 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-be7994ab-c2ea-4d58-89f4-6fec469aeb70 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:33:30.417198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-be7994ab-c2ea-4d58-89f4-6fec469aeb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:30.492601 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 631/1248] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:33:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:33:30.773083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b01a5-8440-4e1a-8ee0-ddabbb2497cb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:30.792716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:30.895463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b01a5-8440-4e1a-8ee0-ddabbb2497cb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:30.937232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad8f2d-9a44-4c37-a37e-627d78194f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:30.937957 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad8f2d-9a44-4c37-a37e-627d78194f88 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:30.938313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad8f2d-9a44-4c37-a37e-627d78194f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:30.938720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad8f2d-9a44-4c37-a37e-627d78194f88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:30.952277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad8f2d-9a44-4c37-a37e-627d78194f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-18ad8f2d-9a44-4c37-a37e-627d78194f88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:33:30.957218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:30.958221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:30.959236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 618/1249] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:33:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:30.969241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7994ab-c2ea-4d58-89f4-6fec469aeb70 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:31.086750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7994ab-c2ea-4d58-89f4-6fec469aeb70 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:31.121116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:31.128333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e5767-74d6-4628-a855-15736eff0396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:31.129097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5767-74d6-4628-a855-15736eff0396 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:31.129343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5767-74d6-4628-a855-15736eff0396 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:31.129614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5767-74d6-4628-a855-15736eff0396 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:31.130402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e5767-74d6-4628-a855-15736eff0396 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:31.130709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e5767-74d6-4628-a855-15736eff0396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:31.132795 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e5767-74d6-4628-a855-15736eff0396 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:31.141801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 632/1250] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:33:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:31.148234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad8f2d-9a44-4c37-a37e-627d78194f88 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:31.250013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:31.284085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad8f2d-9a44-4c37-a37e-627d78194f88 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:31.326246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caada60c-5271-44aa-9fd2-a3f89dbf3068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:31.327116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caada60c-5271-44aa-9fd2-a3f89dbf3068 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:31.327512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caada60c-5271-44aa-9fd2-a3f89dbf3068 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:31.327930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caada60c-5271-44aa-9fd2-a3f89dbf3068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:31.341381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caada60c-5271-44aa-9fd2-a3f89dbf3068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:31.342866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caada60c-5271-44aa-9fd2-a3f89dbf3068 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-caada60c-5271-44aa-9fd2-a3f89dbf3068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:31.358550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:31.364541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:31.365722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:31.365722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:31.365722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:31.365722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 619/1251] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:31.366506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e5767-74d6-4628-a855-15736eff0396 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:31.484086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e5767-74d6-4628-a855-15736eff0396 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:31.522942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e2d6f-8713-43a6-835a-8cabde659845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:31.523505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e2d6f-8713-43a6-835a-8cabde659845 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:31.523706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e2d6f-8713-43a6-835a-8cabde659845 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:31.524038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e2d6f-8713-43a6-835a-8cabde659845 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:31.524621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e2d6f-8713-43a6-835a-8cabde659845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:31.525085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e2d6f-8713-43a6-835a-8cabde659845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:31.526414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e2d6f-8713-43a6-835a-8cabde659845 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:31.539620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 633/1252] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:31.545557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caada60c-5271-44aa-9fd2-a3f89dbf3068 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:31.654680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caada60c-5271-44aa-9fd2-a3f89dbf3068 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:31.683589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:31.692439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c934a4-5f9e-4da6-a126-9cb8681d92a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:31.693101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c934a4-5f9e-4da6-a126-9cb8681d92a2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:31.693379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c934a4-5f9e-4da6-a126-9cb8681d92a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:31.693655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c934a4-5f9e-4da6-a126-9cb8681d92a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:31.702013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c934a4-5f9e-4da6-a126-9cb8681d92a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:31.703786 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c934a4-5f9e-4da6-a126-9cb8681d92a2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c3c934a4-5f9e-4da6-a126-9cb8681d92a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:33:31.716072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:31.717272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:31.718608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:31.718608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:31.718608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:33:31.718608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:33:31.718608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:33:31.718608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:33:31.718608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:33:31.718608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:33:31.718608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:31.718608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:31.718608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 620/1253] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 11:33:31.723260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e2d6f-8713-43a6-835a-8cabde659845 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:31.786820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:31.841649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e2d6f-8713-43a6-835a-8cabde659845 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:31.882692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca1d0e-1983-4878-8656-b1f37091cbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:31.883337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca1d0e-1983-4878-8656-b1f37091cbce None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:31.883658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca1d0e-1983-4878-8656-b1f37091cbce None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:31.883928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca1d0e-1983-4878-8656-b1f37091cbce None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:31.884757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca1d0e-1983-4878-8656-b1f37091cbce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:31.885058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca1d0e-1983-4878-8656-b1f37091cbce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:31.886467 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca1d0e-1983-4878-8656-b1f37091cbce None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:31.906180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 634/1254] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Aug 31 11:33:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:33:31.912521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c934a4-5f9e-4da6-a126-9cb8681d92a2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:32.030330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c934a4-5f9e-4da6-a126-9cb8681d92a2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:32.067338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da5ff7-0d9a-4d44-a394-a09404b46cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:32.067966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5ff7-0d9a-4d44-a394-a09404b46cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:32.068237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5ff7-0d9a-4d44-a394-a09404b46cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:32.068535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5ff7-0d9a-4d44-a394-a09404b46cf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:32.126149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:32.140673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da5ff7-0d9a-4d44-a394-a09404b46cf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:32.142534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da5ff7-0d9a-4d44-a394-a09404b46cf5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:32.265747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 621/1255] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:33:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:33:32.271480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca1d0e-1983-4878-8656-b1f37091cbce None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:32.377215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca1d0e-1983-4878-8656-b1f37091cbce None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:32.392205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:32.415352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c22ab-06f7-4f8c-91be-f5de80d3612c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:32.416008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c22ab-06f7-4f8c-91be-f5de80d3612c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:32.416387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c22ab-06f7-4f8c-91be-f5de80d3612c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:32.416789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c22ab-06f7-4f8c-91be-f5de80d3612c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:32.427071 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c22ab-06f7-4f8c-91be-f5de80d3612c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:32.428575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c22ab-06f7-4f8c-91be-f5de80d3612c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-202c22ab-06f7-4f8c-91be-f5de80d3612c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:32.447043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:32.448247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:32.449355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:32.449355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:32.449355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:32.449355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 635/1256] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:32.454141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da5ff7-0d9a-4d44-a394-a09404b46cf5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:32.551423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:32.565336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da5ff7-0d9a-4d44-a394-a09404b46cf5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:32.604736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c9708-e48e-4037-bfe1-0d92beb9226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:32.605476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c9708-e48e-4037-bfe1-0d92beb9226b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:32.605761 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c9708-e48e-4037-bfe1-0d92beb9226b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:32.606036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c9708-e48e-4037-bfe1-0d92beb9226b None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:32.606690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c9708-e48e-4037-bfe1-0d92beb9226b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:32.606998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c9708-e48e-4037-bfe1-0d92beb9226b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:32.608403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c9708-e48e-4037-bfe1-0d92beb9226b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:32.622907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 622/1257] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:32.629128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c22ab-06f7-4f8c-91be-f5de80d3612c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:32.746352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c22ab-06f7-4f8c-91be-f5de80d3612c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:32.784235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4d22-92e4-4de5-ac86-eb0bf6581bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:32.785018 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4d22-92e4-4de5-ac86-eb0bf6581bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:32.785302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4d22-92e4-4de5-ac86-eb0bf6581bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:32.785571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4d22-92e4-4de5-ac86-eb0bf6581bf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:32.793306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e4d22-92e4-4de5-ac86-eb0bf6581bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:32.794748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e4d22-92e4-4de5-ac86-eb0bf6581bf2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5e5e4d22-92e4-4de5-ac86-eb0bf6581bf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:33:32.807334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:32.813306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:32.814487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:33:32.814487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:33:32.814487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:33:32.814487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:33:32.814487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:33:32.814487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:33:32.814487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:32.814487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:32.814487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 636/1258] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:33:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:32.815122 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c9708-e48e-4037-bfe1-0d92beb9226b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:32.923987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c9708-e48e-4037-bfe1-0d92beb9226b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:32.963603 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbfc05-40e4-45cf-9ac6-da25f412134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:32.964348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbfc05-40e4-45cf-9ac6-da25f412134a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:32.964808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbfc05-40e4-45cf-9ac6-da25f412134a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:32.965071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbfc05-40e4-45cf-9ac6-da25f412134a None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:32.965830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbfc05-40e4-45cf-9ac6-da25f412134a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:32.966173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbfc05-40e4-45cf-9ac6-da25f412134a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:32.967828 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbfc05-40e4-45cf-9ac6-da25f412134a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:32.981204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 623/1259] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:33:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:33:32.987372 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4d22-92e4-4de5-ac86-eb0bf6581bf2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:33.016954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:33.105021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4d22-92e4-4de5-ac86-eb0bf6581bf2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:33.142959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5864d-8051-41e3-b856-730940dcb372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:33.143711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5864d-8051-41e3-b856-730940dcb372 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:33.144075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5864d-8051-41e3-b856-730940dcb372 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:33.144455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5864d-8051-41e3-b856-730940dcb372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:33.145248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5864d-8051-41e3-b856-730940dcb372 None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=f9114a71a3a247188b263d87b6cb12c6) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:33.145559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5864d-8051-41e3-b856-730940dcb372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:33.147081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5864d-8051-41e3-b856-730940dcb372 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:33.154034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 637/1260] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:33:32 2022] GET /identity/v3/role_assignments?user.id=f9114a71a3a247188b263d87b6cb12c6&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:33:35.051440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b4232-48fe-4ac5-82a1-28e9461469f4 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:35.051940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b4232-48fe-4ac5-82a1-28e9461469f4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:35.051940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b4232-48fe-4ac5-82a1-28e9461469f4 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:35.053271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 624/1261] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:33:35.059100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f39c69-eee3-455f-b31b-324f2e95700a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:35.059589 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f39c69-eee3-455f-b31b-324f2e95700a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:35.059683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f39c69-eee3-455f-b31b-324f2e95700a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:35.078463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-76f39c69-eee3-455f-b31b-324f2e95700a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:33:35.079631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-76f39c69-eee3-455f-b31b-324f2e95700a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:35.145685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 638/1262] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:33:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:33:35.437030 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b4232-48fe-4ac5-82a1-28e9461469f4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:35.448788 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:35.535865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b4232-48fe-4ac5-82a1-28e9461469f4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:35.568726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782e97-7a12-4ec4-b431-6367c8ec521c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:35.570115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782e97-7a12-4ec4-b431-6367c8ec521c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:35.570396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782e97-7a12-4ec4-b431-6367c8ec521c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:35.570668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782e97-7a12-4ec4-b431-6367c8ec521c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:35.582356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10782e97-7a12-4ec4-b431-6367c8ec521c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-10782e97-7a12-4ec4-b431-6367c8ec521c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:33:35.587737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:35.588910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:35.590015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 625/1263] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:33:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:35.594772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f39c69-eee3-455f-b31b-324f2e95700a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:35.612726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:35.692135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f39c69-eee3-455f-b31b-324f2e95700a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:35.728479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d0d0c-3165-430f-bb85-2ffbe7b0042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:35.729099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d0d0c-3165-430f-bb85-2ffbe7b0042c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:35.729378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d0d0c-3165-430f-bb85-2ffbe7b0042c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:35.729648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d0d0c-3165-430f-bb85-2ffbe7b0042c None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:35.730297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d0d0c-3165-430f-bb85-2ffbe7b0042c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:35.730597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d0d0c-3165-430f-bb85-2ffbe7b0042c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:35.731976 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d0d0c-3165-430f-bb85-2ffbe7b0042c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:35.739726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 639/1264] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:33:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:35.745388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782e97-7a12-4ec4-b431-6367c8ec521c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:35.810318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:35.842834 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782e97-7a12-4ec4-b431-6367c8ec521c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:35.876149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e23cf6-f237-49f8-a10c-4bb5c6915518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:35.876801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e23cf6-f237-49f8-a10c-4bb5c6915518 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:35.877069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e23cf6-f237-49f8-a10c-4bb5c6915518 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:35.877344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e23cf6-f237-49f8-a10c-4bb5c6915518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:35.887006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e23cf6-f237-49f8-a10c-4bb5c6915518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:35.888441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e23cf6-f237-49f8-a10c-4bb5c6915518 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-68e23cf6-f237-49f8-a10c-4bb5c6915518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:35.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:35.904394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:35.905748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:35.905748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:35.905748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:35.905748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 626/1265] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:35.910038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d0d0c-3165-430f-bb85-2ffbe7b0042c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:35.989349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:36.007936 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d0d0c-3165-430f-bb85-2ffbe7b0042c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:36.040849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe160012-d915-4dc2-9e02-de9ce292d2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:36.041444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe160012-d915-4dc2-9e02-de9ce292d2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:36.041747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe160012-d915-4dc2-9e02-de9ce292d2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:36.042019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe160012-d915-4dc2-9e02-de9ce292d2d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:36.042649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe160012-d915-4dc2-9e02-de9ce292d2d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:36.042951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe160012-d915-4dc2-9e02-de9ce292d2d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:36.044295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe160012-d915-4dc2-9e02-de9ce292d2d2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:36.055924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 640/1266] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:36.061559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e23cf6-f237-49f8-a10c-4bb5c6915518 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:36.157550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e23cf6-f237-49f8-a10c-4bb5c6915518 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:36.178047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:36.195454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1b45c-d058-4a84-b7db-cacc20df17fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:36.196236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1b45c-d058-4a84-b7db-cacc20df17fd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:36.196624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1b45c-d058-4a84-b7db-cacc20df17fd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:36.196970 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1b45c-d058-4a84-b7db-cacc20df17fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:36.206158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1b45c-d058-4a84-b7db-cacc20df17fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:36.207777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1b45c-d058-4a84-b7db-cacc20df17fd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-20e1b45c-d058-4a84-b7db-cacc20df17fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:33:36.220595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:36.221816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:36.222996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:36.222996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:36.222996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:33:36.222996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:33:36.222996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:33:36.222996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:33:36.222996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:33:36.222996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:33:36.222996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:36.222996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:36.222996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 627/1267] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:36.228377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe160012-d915-4dc2-9e02-de9ce292d2d2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:36.324583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe160012-d915-4dc2-9e02-de9ce292d2d2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:36.357010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b4671-10a7-422e-be7d-d242615f9626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:36.357593 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b4671-10a7-422e-be7d-d242615f9626 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:36.357869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b4671-10a7-422e-be7d-d242615f9626 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:36.358138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b4671-10a7-422e-be7d-d242615f9626 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:36.358846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b4671-10a7-422e-be7d-d242615f9626 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:36.359145 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b4671-10a7-422e-be7d-d242615f9626 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:36.360559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b4671-10a7-422e-be7d-d242615f9626 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:36.369038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:36.378270 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 641/1268] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Aug 31 11:33:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:36.384811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1b45c-d058-4a84-b7db-cacc20df17fd None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:36.486374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1b45c-d058-4a84-b7db-cacc20df17fd None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:36.519536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85a78a-edab-4542-9e03-6d285bd8a06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:36.520145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85a78a-edab-4542-9e03-6d285bd8a06c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:36.520428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85a78a-edab-4542-9e03-6d285bd8a06c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:36.520734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85a78a-edab-4542-9e03-6d285bd8a06c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:36.531799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85a78a-edab-4542-9e03-6d285bd8a06c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:36.533278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85a78a-edab-4542-9e03-6d285bd8a06c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-eb85a78a-edab-4542-9e03-6d285bd8a06c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:36.547347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:36.548441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:36.549742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:36.549742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:36.549742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:36.549742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 628/1269] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:36.554240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b4671-10a7-422e-be7d-d242615f9626 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:36.651447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b4671-10a7-422e-be7d-d242615f9626 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:36.685815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7688a-4ad8-482e-9f8a-24c37239329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:36.686413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7688a-4ad8-482e-9f8a-24c37239329d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:36.686694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7688a-4ad8-482e-9f8a-24c37239329d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:36.686965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7688a-4ad8-482e-9f8a-24c37239329d None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:36.687596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7688a-4ad8-482e-9f8a-24c37239329d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:36.687899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7688a-4ad8-482e-9f8a-24c37239329d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:36.689280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7688a-4ad8-482e-9f8a-24c37239329d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:36.701655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 642/1270] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:36.707277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85a78a-edab-4542-9e03-6d285bd8a06c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:36.738411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:36.805896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85a78a-edab-4542-9e03-6d285bd8a06c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:36.838610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f2208-a10e-4a5c-a1f3-942f3aa4a8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:36.839216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f2208-a10e-4a5c-a1f3-942f3aa4a8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:36.839497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f2208-a10e-4a5c-a1f3-942f3aa4a8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:36.839771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f2208-a10e-4a5c-a1f3-942f3aa4a8e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:36.846171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f2208-a10e-4a5c-a1f3-942f3aa4a8e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:36.847600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f2208-a10e-4a5c-a1f3-942f3aa4a8e9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-035f2208-a10e-4a5c-a1f3-942f3aa4a8e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:33:36.858685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:36.859791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:36.860893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:33:36.860893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:33:36.860893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:33:36.860893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:33:36.860893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:33:36.860893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:33:36.860893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:36.860893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:36.860893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 629/1271] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:36.865530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7688a-4ad8-482e-9f8a-24c37239329d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:36.921896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:36.962906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7688a-4ad8-482e-9f8a-24c37239329d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:36.995675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c4fbc-916c-40df-b79c-a19246165195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:36.996258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c4fbc-916c-40df-b79c-a19246165195 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:36.996557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c4fbc-916c-40df-b79c-a19246165195 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:36.996826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c4fbc-916c-40df-b79c-a19246165195 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:36.997497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c4fbc-916c-40df-b79c-a19246165195 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:36.997796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c4fbc-916c-40df-b79c-a19246165195 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:36.999197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c4fbc-916c-40df-b79c-a19246165195 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:37.010725 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 643/1272] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:37.016642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f2208-a10e-4a5c-a1f3-942f3aa4a8e9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:37.106023 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:37.114721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f2208-a10e-4a5c-a1f3-942f3aa4a8e9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:37.147688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10f9a8-4bf1-4462-961f-07c6b2941cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:37.148379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10f9a8-4bf1-4462-961f-07c6b2941cd9 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:37.148689 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10f9a8-4bf1-4462-961f-07c6b2941cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:37.148967 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10f9a8-4bf1-4462-961f-07c6b2941cd9 None admin] PATH_INFO: `/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/f9114a71a3a247188b263d87b6cb12c6/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:37.150833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10f9a8-4bf1-4462-961f-07c6b2941cd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b4dae6d2b2345dc9259776a4237e4e6, user_id=f9114a71a3a247188b263d87b6cb12c6, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:37.152269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10f9a8-4bf1-4462-961f-07c6b2941cd9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:37.164258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 630/1273] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Aug 31 11:33:37 2022] PUT /identity/v3/projects/3b4dae6d2b2345dc9259776a4237e4e6/users/f9114a71a3a247188b263d87b6cb12c6/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:33:39.063096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f653e05-b33b-4788-bfd3-2d7ef926e21a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:39.063596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f653e05-b33b-4788-bfd3-2d7ef926e21a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:39.063596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f653e05-b33b-4788-bfd3-2d7ef926e21a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:39.064917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 644/1274] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:33:39.070335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d722a-ac6f-4bd6-b2e3-8e25c029b616 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:39.070717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d722a-ac6f-4bd6-b2e3-8e25c029b616 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:39.070882 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d722a-ac6f-4bd6-b2e3-8e25c029b616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:39.089863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-459d722a-ac6f-4bd6-b2e3-8e25c029b616 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:33:39.091057 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-459d722a-ac6f-4bd6-b2e3-8e25c029b616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:39.156631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 631/1275] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:33:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:33:39.437936 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f653e05-b33b-4788-bfd3-2d7ef926e21a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:39.536337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f653e05-b33b-4788-bfd3-2d7ef926e21a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:39.554249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:39.579592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef141398-cdb5-4e7c-a5e4-6c30122f1d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:39.580216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef141398-cdb5-4e7c-a5e4-6c30122f1d91 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:39.580611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef141398-cdb5-4e7c-a5e4-6c30122f1d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:39.580886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef141398-cdb5-4e7c-a5e4-6c30122f1d91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:39.591762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef141398-cdb5-4e7c-a5e4-6c30122f1d91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ef141398-cdb5-4e7c-a5e4-6c30122f1d91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 11:33:39.596597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:39.597703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:39.598909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 645/1276] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Aug 31 11:33:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:39.604050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d722a-ac6f-4bd6-b2e3-8e25c029b616 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:39.684653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:39.704165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d722a-ac6f-4bd6-b2e3-8e25c029b616 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:39.737000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28df018-caa1-4481-8565-31c62c8369af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:39.737601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28df018-caa1-4481-8565-31c62c8369af None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:39.737883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28df018-caa1-4481-8565-31c62c8369af None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:39.738156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28df018-caa1-4481-8565-31c62c8369af None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:39.738815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28df018-caa1-4481-8565-31c62c8369af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:39.739118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28df018-caa1-4481-8565-31c62c8369af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:39.740524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28df018-caa1-4481-8565-31c62c8369af None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:39.748793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 632/1277] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Aug 31 11:33:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:39.755018 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef141398-cdb5-4e7c-a5e4-6c30122f1d91 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:39.852667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef141398-cdb5-4e7c-a5e4-6c30122f1d91 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:39.887041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7c43d-9223-4d5e-904a-b58f8e1805a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:39.887646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7c43d-9223-4d5e-904a-b58f8e1805a7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:39.887924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7c43d-9223-4d5e-904a-b58f8e1805a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:39.888192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7c43d-9223-4d5e-904a-b58f8e1805a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:39.898008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7c43d-9223-4d5e-904a-b58f8e1805a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:39.899429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7c43d-9223-4d5e-904a-b58f8e1805a7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-23a7c43d-9223-4d5e-904a-b58f8e1805a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:39.913710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:39.914832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:39.916007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:39.916007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:39.916007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:39.916007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 646/1278] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:39.922435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28df018-caa1-4481-8565-31c62c8369af None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:40.021162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28df018-caa1-4481-8565-31c62c8369af None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:40.053183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bff78-47e6-404b-b980-7875b3869f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:40.054316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bff78-47e6-404b-b980-7875b3869f08 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:40.054592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bff78-47e6-404b-b980-7875b3869f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:40.054865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bff78-47e6-404b-b980-7875b3869f08 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:40.055498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bff78-47e6-404b-b980-7875b3869f08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:40.055800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bff78-47e6-404b-b980-7875b3869f08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:40.057187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bff78-47e6-404b-b980-7875b3869f08 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:40.068415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 633/1279] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:40.074788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7c43d-9223-4d5e-904a-b58f8e1805a7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:40.107288 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:40.180763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7c43d-9223-4d5e-904a-b58f8e1805a7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:40.219122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5ddad-37ed-4771-8189-8b9ba735be2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:40.219735 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5ddad-37ed-4771-8189-8b9ba735be2d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:40.220008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5ddad-37ed-4771-8189-8b9ba735be2d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:40.220298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5ddad-37ed-4771-8189-8b9ba735be2d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:40.227553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5ddad-37ed-4771-8189-8b9ba735be2d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:40.229006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5ddad-37ed-4771-8189-8b9ba735be2d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-44c5ddad-37ed-4771-8189-8b9ba735be2d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 11:33:40.239690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:40.240838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:40.242226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:40.242226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:40.242226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 11:33:40.242226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 11:33:40.242226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 11:33:40.242226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 11:33:40.242226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 11:33:40.242226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 11:33:40.242226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 11:33:40.242226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:40.242226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 647/1280] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:40.246653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bff78-47e6-404b-b980-7875b3869f08 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:40.250371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:40.344249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bff78-47e6-404b-b980-7875b3869f08 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:40.376612 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae611c-abd0-49d5-8c34-07d2a03d468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:40.377201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae611c-abd0-49d5-8c34-07d2a03d468e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:40.377479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae611c-abd0-49d5-8c34-07d2a03d468e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:40.377749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae611c-abd0-49d5-8c34-07d2a03d468e None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:40.378464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae611c-abd0-49d5-8c34-07d2a03d468e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:40.378770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae611c-abd0-49d5-8c34-07d2a03d468e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:40.380175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae611c-abd0-49d5-8c34-07d2a03d468e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:40.396773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 634/1281] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Aug 31 11:33:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:40.402594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5ddad-37ed-4771-8189-8b9ba735be2d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:40.492915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:40.504980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5ddad-37ed-4771-8189-8b9ba735be2d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:40.538621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21275f-998e-4c26-b308-d77ae1c75454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:40.539221 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21275f-998e-4c26-b308-d77ae1c75454 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:40.539502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21275f-998e-4c26-b308-d77ae1c75454 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:40.539778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21275f-998e-4c26-b308-d77ae1c75454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:40.605411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21275f-998e-4c26-b308-d77ae1c75454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:40.606774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21275f-998e-4c26-b308-d77ae1c75454 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:40.693212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:40.704478 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 648/1282] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Aug 31 11:33:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 11:33:40.710370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae611c-abd0-49d5-8c34-07d2a03d468e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:40.771881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:40.810705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae611c-abd0-49d5-8c34-07d2a03d468e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:40.843404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2520a4-42cd-4e8c-baac-18533cce65d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:40.844021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2520a4-42cd-4e8c-baac-18533cce65d3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:40.844302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2520a4-42cd-4e8c-baac-18533cce65d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:40.844604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2520a4-42cd-4e8c-baac-18533cce65d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:40.854874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2520a4-42cd-4e8c-baac-18533cce65d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:40.856377 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2520a4-42cd-4e8c-baac-18533cce65d3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-be2520a4-42cd-4e8c-baac-18533cce65d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 11:33:40.871230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:40.872367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 11:33:40.873514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 11:33:40.873514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 11:33:40.873514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:40.873514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 635/1283] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:40.878054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21275f-998e-4c26-b308-d77ae1c75454 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:40.973243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21275f-998e-4c26-b308-d77ae1c75454 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:41.006087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fc283-e0e4-4f8b-a362-5ccf38a27f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:41.006709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fc283-e0e4-4f8b-a362-5ccf38a27f40 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:41.006985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fc283-e0e4-4f8b-a362-5ccf38a27f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:41.007249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fc283-e0e4-4f8b-a362-5ccf38a27f40 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:41.007874 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fc283-e0e4-4f8b-a362-5ccf38a27f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:41.008170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fc283-e0e4-4f8b-a362-5ccf38a27f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:41.009545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fc283-e0e4-4f8b-a362-5ccf38a27f40 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:41.020154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 649/1284] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Aug 31 11:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:41.025772 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2520a4-42cd-4e8c-baac-18533cce65d3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:41.124901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2520a4-42cd-4e8c-baac-18533cce65d3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:41.138750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:41.159573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d848fc-4a76-4c0a-93f9-f66d0733bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:41.160197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d848fc-4a76-4c0a-93f9-f66d0733bcde None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:41.160480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d848fc-4a76-4c0a-93f9-f66d0733bcde None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:41.160784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d848fc-4a76-4c0a-93f9-f66d0733bcde None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:41.167944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d848fc-4a76-4c0a-93f9-f66d0733bcde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:41.169440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d848fc-4a76-4c0a-93f9-f66d0733bcde None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-90d848fc-4a76-4c0a-93f9-f66d0733bcde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 11:33:41.182336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:41.183403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:41.184959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 11:33:41.184959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 11:33:41.184959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 11:33:41.184959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 11:33:41.184959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 11:33:41.184959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 11:33:41.184959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 11:33:41.184959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 31 11:33:41.184959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 636/1285] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Aug 31 11:33:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:41.189655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fc283-e0e4-4f8b-a362-5ccf38a27f40 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:41.242425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:41.291189 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fc283-e0e4-4f8b-a362-5ccf38a27f40 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:41.324057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd191b29-057a-4cfe-b075-7876d09a72c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:41.324692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191b29-057a-4cfe-b075-7876d09a72c9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:41.324981 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191b29-057a-4cfe-b075-7876d09a72c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:41.325255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191b29-057a-4cfe-b075-7876d09a72c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:41.325930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd191b29-057a-4cfe-b075-7876d09a72c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:41.326236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd191b29-057a-4cfe-b075-7876d09a72c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:41.327645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd191b29-057a-4cfe-b075-7876d09a72c9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:41.340329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 650/1286] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Aug 31 11:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:41.346406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d848fc-4a76-4c0a-93f9-f66d0733bcde None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:41.441725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d848fc-4a76-4c0a-93f9-f66d0733bcde None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:41.473909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b6702-0c50-4795-aa99-9852c40f4bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:41.474494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b6702-0c50-4795-aa99-9852c40f4bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:41.474771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b6702-0c50-4795-aa99-9852c40f4bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:41.475040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b6702-0c50-4795-aa99-9852c40f4bc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:41.476350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b6702-0c50-4795-aa99-9852c40f4bc1 None admin] RBAC: Adding query filter params (role.id=6cb25aac3b3e48f888f9a84dcef654e6, scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6, user.id=f9114a71a3a247188b263d87b6cb12c6) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:41.476684 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b6702-0c50-4795-aa99-9852c40f4bc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:41.478111 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b6702-0c50-4795-aa99-9852c40f4bc1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:41.483592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 637/1287] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Aug 31 11:33:41 2022] GET /identity/v3/role_assignments?user.id=f9114a71a3a247188b263d87b6cb12c6&scope.project.id=3b4dae6d2b2345dc9259776a4237e4e6&role.id=6cb25aac3b3e48f888f9a84dcef654e6 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:43.392043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b86611-b7c9-4ef6-b978-f8325a434b01 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:43.392626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b86611-b7c9-4ef6-b978-f8325a434b01 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:43.392626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b86611-b7c9-4ef6-b978-f8325a434b01 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:43.393927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 651/1288] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:33:43.399728 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14503544-0e58-4ea6-85c0-be870ca4a42a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:43.400140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14503544-0e58-4ea6-85c0-be870ca4a42a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:43.400290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14503544-0e58-4ea6-85c0-be870ca4a42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:43.418969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-14503544-0e58-4ea6-85c0-be870ca4a42a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:33:43.420164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-14503544-0e58-4ea6-85c0-be870ca4a42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:43.449239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:43.488028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 638/1289] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:33:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:33:43.756106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b86611-b7c9-4ef6-b978-f8325a434b01 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:43.853217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b86611-b7c9-4ef6-b978-f8325a434b01 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:43.863398 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:43.886679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ef747-0313-45d5-ab3d-1b070d564dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:43.887284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ef747-0313-45d5-ab3d-1b070d564dbc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:43.887559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ef747-0313-45d5-ab3d-1b070d564dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:43.887826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ef747-0313-45d5-ab3d-1b070d564dbc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:43.888392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ef747-0313-45d5-ab3d-1b070d564dbc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:43.889762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ef747-0313-45d5-ab3d-1b070d564dbc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-521ef747-0313-45d5-ab3d-1b070d564dbc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:43.898276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:43.899564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:43.900750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 652/1290] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Aug 31 11:33:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:43.905267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14503544-0e58-4ea6-85c0-be870ca4a42a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:44.001150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14503544-0e58-4ea6-85c0-be870ca4a42a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:44.033371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4474d32-a613-4dc9-9299-a8a9ecebeaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:44.033964 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4474d32-a613-4dc9-9299-a8a9ecebeaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:44.034247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4474d32-a613-4dc9-9299-a8a9ecebeaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:44.034520 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4474d32-a613-4dc9-9299-a8a9ecebeaa9 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:44.035140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4474d32-a613-4dc9-9299-a8a9ecebeaa9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:44.035442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4474d32-a613-4dc9-9299-a8a9ecebeaa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:44.036829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4474d32-a613-4dc9-9299-a8a9ecebeaa9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:44.042100 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 639/1291] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Aug 31 11:33:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:44.047681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ef747-0313-45d5-ab3d-1b070d564dbc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:44.145334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ef747-0313-45d5-ab3d-1b070d564dbc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:44.179931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e17981-c4da-4240-b8eb-9c46072847b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:44.180660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e17981-c4da-4240-b8eb-9c46072847b1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:44.180978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e17981-c4da-4240-b8eb-9c46072847b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:44.181249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e17981-c4da-4240-b8eb-9c46072847b1 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:44.181967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e17981-c4da-4240-b8eb-9c46072847b1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:44.182312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e17981-c4da-4240-b8eb-9c46072847b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:44.183791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e17981-c4da-4240-b8eb-9c46072847b1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:44.189242 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 653/1292] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Aug 31 11:33:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:46.106079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82172caf-9a37-4ab3-bc10-cc2b39853704 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:46.106601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82172caf-9a37-4ab3-bc10-cc2b39853704 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:46.106722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82172caf-9a37-4ab3-bc10-cc2b39853704 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:46.107977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 640/1293] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:33:46.112942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2739ebb-f14c-454b-b176-612915adb612 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:46.113319 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2739ebb-f14c-454b-b176-612915adb612 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:46.113483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2739ebb-f14c-454b-b176-612915adb612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:46.131119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e2739ebb-f14c-454b-b176-612915adb612 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:33:46.132851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e2739ebb-f14c-454b-b176-612915adb612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:46.161470 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:46.199017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 654/1294] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:33:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:33:46.468156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82172caf-9a37-4ab3-bc10-cc2b39853704 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:46.499476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:46.568501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82172caf-9a37-4ab3-bc10-cc2b39853704 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:46.614082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea81d0-4bcc-4292-87d0-bc60714d173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:46.614690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea81d0-4bcc-4292-87d0-bc60714d173c None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:46.614975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea81d0-4bcc-4292-87d0-bc60714d173c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:46.615252 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea81d0-4bcc-4292-87d0-bc60714d173c None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:46.616320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea81d0-4bcc-4292-87d0-bc60714d173c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:46.617471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea81d0-4bcc-4292-87d0-bc60714d173c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:46.631732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 641/1295] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Aug 31 11:33:46 2022] POST /identity/v3/services => generated 246 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:33:48.524067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa239e7e-f3f3-4fd9-b3e0-5c5e0c9da37e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:48.524624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa239e7e-f3f3-4fd9-b3e0-5c5e0c9da37e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:48.524624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa239e7e-f3f3-4fd9-b3e0-5c5e0c9da37e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:48.525954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 655/1296] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:33:48.531142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c231f9-a7d6-460a-8db8-e4488f830ebf None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:48.531560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c231f9-a7d6-460a-8db8-e4488f830ebf None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:48.531699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c231f9-a7d6-460a-8db8-e4488f830ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:48.550085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a9c231f9-a7d6-460a-8db8-e4488f830ebf None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:33:48.551263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a9c231f9-a7d6-460a-8db8-e4488f830ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:48.623322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 642/1297] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:33:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:33:48.888334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa239e7e-f3f3-4fd9-b3e0-5c5e0c9da37e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:48.986551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa239e7e-f3f3-4fd9-b3e0-5c5e0c9da37e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:49.019292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80df50-9098-41f8-a176-ed20d0595d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:49.019900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80df50-9098-41f8-a176-ed20d0595d2b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:49.020179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80df50-9098-41f8-a176-ed20d0595d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:49.020449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80df50-9098-41f8-a176-ed20d0595d2b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:49.021052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80df50-9098-41f8-a176-ed20d0595d2b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:49.022649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80df50-9098-41f8-a176-ed20d0595d2b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0e80df50-9098-41f8-a176-ed20d0595d2b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:49.031175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:49.032345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:49.033611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 656/1298] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Aug 31 11:33:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:49.038688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c231f9-a7d6-460a-8db8-e4488f830ebf None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:49.056878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:49.139338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c231f9-a7d6-460a-8db8-e4488f830ebf None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:49.174578 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f598ae-0c14-4a6d-88ed-2aad52239d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:49.175192 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f598ae-0c14-4a6d-88ed-2aad52239d25 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:49.175475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f598ae-0c14-4a6d-88ed-2aad52239d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:49.175750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f598ae-0c14-4a6d-88ed-2aad52239d25 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:49.176379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f598ae-0c14-4a6d-88ed-2aad52239d25 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:49.176720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f598ae-0c14-4a6d-88ed-2aad52239d25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:49.178119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f598ae-0c14-4a6d-88ed-2aad52239d25 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:49.183808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 643/1299] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Aug 31 11:33:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:49.189735 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80df50-9098-41f8-a176-ed20d0595d2b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:49.221545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:49.292318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80df50-9098-41f8-a176-ed20d0595d2b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:49.326293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30855854-a0f5-4271-98e4-afaea344c0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:49.326883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30855854-a0f5-4271-98e4-afaea344c0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:49.327168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30855854-a0f5-4271-98e4-afaea344c0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:49.327440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30855854-a0f5-4271-98e4-afaea344c0e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:49.328133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30855854-a0f5-4271-98e4-afaea344c0e2 None admin] RBAC: Adding query filter params (interface=public, service_id=88e19ca146e54b4aad4ec6315ec2d117, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:49.328443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30855854-a0f5-4271-98e4-afaea344c0e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:49.329833 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30855854-a0f5-4271-98e4-afaea344c0e2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:49.335286 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 657/1300] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Aug 31 11:33:49 2022] GET /identity/v3/endpoints?service_id=88e19ca146e54b4aad4ec6315ec2d117&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:49.340923 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f598ae-0c14-4a6d-88ed-2aad52239d25 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:49.422215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:49.440044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f598ae-0c14-4a6d-88ed-2aad52239d25 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:49.473934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c9f2b-dd0d-4200-8552-c483ffd10615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:49.474529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c9f2b-dd0d-4200-8552-c483ffd10615 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:49.474812 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c9f2b-dd0d-4200-8552-c483ffd10615 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:49.475660 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c9f2b-dd0d-4200-8552-c483ffd10615 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:49.476766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c9f2b-dd0d-4200-8552-c483ffd10615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:49.478136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c9f2b-dd0d-4200-8552-c483ffd10615 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:49.484791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 644/1301] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Aug 31 11:33:49 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:33:51.365558 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a0036-5081-40b8-a985-762162ce0ad7 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:51.366317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a0036-5081-40b8-a985-762162ce0ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:51.366317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a0036-5081-40b8-a985-762162ce0ad7 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:51.367579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 658/1302] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:33:51.372559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b700863-6f6f-4f12-a47a-016005e59f29 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:51.372945 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b700863-6f6f-4f12-a47a-016005e59f29 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:51.373120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b700863-6f6f-4f12-a47a-016005e59f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:51.391412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5b700863-6f6f-4f12-a47a-016005e59f29 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:33:51.392631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5b700863-6f6f-4f12-a47a-016005e59f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:51.457036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 645/1303] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:33:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 11:33:51.735974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a0036-5081-40b8-a985-762162ce0ad7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:51.834686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:51.841673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a0036-5081-40b8-a985-762162ce0ad7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:51.875021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e19396-c094-4d38-aba9-5cea80402f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:51.875710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e19396-c094-4d38-aba9-5cea80402f04 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:51.875992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e19396-c094-4d38-aba9-5cea80402f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:51.876313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e19396-c094-4d38-aba9-5cea80402f04 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:51.877007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e19396-c094-4d38-aba9-5cea80402f04 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:51.878375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e19396-c094-4d38-aba9-5cea80402f04 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-72e19396-c094-4d38-aba9-5cea80402f04 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 11:33:51.887731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 11:33:51.888916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 31 11:33:51.889912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 659/1304] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Aug 31 11:33:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 11:33:51.894810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b700863-6f6f-4f12-a47a-016005e59f29 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:51.970081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:51.993828 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b700863-6f6f-4f12-a47a-016005e59f29 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:52.026276 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a453e01-2835-4e3c-a269-907f7b312182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:52.026878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a453e01-2835-4e3c-a269-907f7b312182 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:52.027163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a453e01-2835-4e3c-a269-907f7b312182 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:52.027440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a453e01-2835-4e3c-a269-907f7b312182 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:52.028072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a453e01-2835-4e3c-a269-907f7b312182 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:33:52.028379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a453e01-2835-4e3c-a269-907f7b312182 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:52.029791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a453e01-2835-4e3c-a269-907f7b312182 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:52.034753 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 646/1305] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Aug 31 11:33:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:33:52.041110 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e19396-c094-4d38-aba9-5cea80402f04 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:52.139432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e19396-c094-4d38-aba9-5cea80402f04 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:52.174512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf41807-c0f6-41e0-93c5-b985ab9e7599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:52.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf41807-c0f6-41e0-93c5-b985ab9e7599 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:52.175978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf41807-c0f6-41e0-93c5-b985ab9e7599 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:52.176294 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf41807-c0f6-41e0-93c5-b985ab9e7599 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:52.177531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf41807-c0f6-41e0-93c5-b985ab9e7599 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:52.179659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf41807-c0f6-41e0-93c5-b985ab9e7599 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:52.205836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 660/1306] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:33:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:34:02.032278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=30 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:34:02.200435 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:34:56.473308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4dc67-10af-44df-b6bb-3f48525e8cc8 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:34:56.473773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4dc67-10af-44df-b6bb-3f48525e8cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:34:56.473932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4dc67-10af-44df-b6bb-3f48525e8cc8 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:34:56.479934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 647/1307] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:34:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:34:56.481928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17f10f-7d64-4a2a-979b-bb957ec1033c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:34:56.482904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17f10f-7d64-4a2a-979b-bb957ec1033c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:34:56.483559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17f10f-7d64-4a2a-979b-bb957ec1033c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:34:56.582530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-db17f10f-7d64-4a2a-979b-bb957ec1033c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:34:56.583731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-db17f10f-7d64-4a2a-979b-bb957ec1033c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:34:56.706289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 661/1308] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:34:57.005042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2587-9bd4-4c1d-ba5b-20a8eb52ca85 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:34:57.005372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2587-9bd4-4c1d-ba5b-20a8eb52ca85 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:34:57.006164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2587-9bd4-4c1d-ba5b-20a8eb52ca85 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:34:57.007692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 648/1309] 10.222.0.60 () {54 vars in 965 bytes} [Wed Aug 31 11:34:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:34:57.013960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c9355-b8ae-4943-91de-ce393c780092 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:34:57.014414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c9355-b8ae-4943-91de-ce393c780092 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:34:57.014770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c9355-b8ae-4943-91de-ce393c780092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:34:57.111302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b81c9355-b8ae-4943-91de-ce393c780092 None None] MFA Rules not processed for user `30546ba7f5614f878a9ba322c7c55ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:34:57.112703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b81c9355-b8ae-4943-91de-ce393c780092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:34:57.137103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:34:57.202663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 662/1310] 10.222.0.60 () {58 vars in 1062 bytes} [Wed Aug 31 11:34:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:34:57.212009 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd2587-9bd4-4c1d-ba5b-20a8eb52ca85 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:34:57.213962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-adfd2587-9bd4-4c1d-ba5b-20a8eb52ca85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:34:57.335678 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd2587-9bd4-4c1d-ba5b-20a8eb52ca85 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:34:57.416098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c778462-e663-4882-9811-3a7c5c062e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:34:57.416736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c778462-e663-4882-9811-3a7c5c062e02 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:34:57.417019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c778462-e663-4882-9811-3a7c5c062e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:34:57.417297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c778462-e663-4882-9811-3a7c5c062e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:34:57.419019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2c778462-e663-4882-9811-3a7c5c062e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:34:57.478800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:34:57.538468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c778462-e663-4882-9811-3a7c5c062e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:34:57.539916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c778462-e663-4882-9811-3a7c5c062e02 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:34:57.560749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 649/1311] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:34:59.803693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5dceb-718c-4c1b-bda7-9182eeb72ff4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:34:59.804243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5dceb-718c-4c1b-bda7-9182eeb72ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:34:59.804243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5dceb-718c-4c1b-bda7-9182eeb72ff4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:34:59.805789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 663/1312] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:34:59.810740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f7241-2d45-4815-8961-e9eb108e7e2e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:34:59.811166 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f7241-2d45-4815-8961-e9eb108e7e2e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:34:59.811295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f7241-2d45-4815-8961-e9eb108e7e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:34:59.897801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7b5f7241-2d45-4815-8961-e9eb108e7e2e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:34:59.899023 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7b5f7241-2d45-4815-8961-e9eb108e7e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:34:59.996145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 650/1313] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:00.278994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5dceb-718c-4c1b-bda7-9182eeb72ff4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:00.280583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-68b5dceb-718c-4c1b-bda7-9182eeb72ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:00.388705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5dceb-718c-4c1b-bda7-9182eeb72ff4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:00.462559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:00.464963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d8f13-b21d-455f-b6b9-4299490a5485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:00.465597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d8f13-b21d-455f-b6b9-4299490a5485 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:00.466069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d8f13-b21d-455f-b6b9-4299490a5485 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:00.466507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d8f13-b21d-455f-b6b9-4299490a5485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:00.468290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-551d8f13-b21d-455f-b6b9-4299490a5485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:00.578633 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d8f13-b21d-455f-b6b9-4299490a5485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:00.580072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d8f13-b21d-455f-b6b9-4299490a5485 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:00.598634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 664/1314] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:02.832969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de50c4-3530-4622-9b72-a74154aa8b93 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:02.833500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de50c4-3530-4622-9b72-a74154aa8b93 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:02.833500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de50c4-3530-4622-9b72-a74154aa8b93 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:02.835945 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 651/1315] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:02.840959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5ff6a-aa14-4d70-b33f-4a7951a3048c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:02.841398 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5ff6a-aa14-4d70-b33f-4a7951a3048c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:02.841496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5ff6a-aa14-4d70-b33f-4a7951a3048c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:02.909272 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e7f5ff6a-aa14-4d70-b33f-4a7951a3048c None None] MFA Rules not processed for user `8e67b785954f4fd4b095552017407c73`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:02.910506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e7f5ff6a-aa14-4d70-b33f-4a7951a3048c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:02.962502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:03.006413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 665/1316] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:03.307183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de50c4-3530-4622-9b72-a74154aa8b93 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:03.308760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-38de50c4-3530-4622-9b72-a74154aa8b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:03.319686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:03.415965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de50c4-3530-4622-9b72-a74154aa8b93 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:03.489584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de472128-b6e2-4272-9dcb-37f823890d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:03.490202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de472128-b6e2-4272-9dcb-37f823890d80 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:03.490482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de472128-b6e2-4272-9dcb-37f823890d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:03.490747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de472128-b6e2-4272-9dcb-37f823890d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:03.492340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-de472128-b6e2-4272-9dcb-37f823890d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:03.553737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:03.601432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de472128-b6e2-4272-9dcb-37f823890d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:03.602901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de472128-b6e2-4272-9dcb-37f823890d80 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:03.622156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 652/1317] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:03 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:06.110484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c6f60-0c44-4ee1-ae86-cb304cc21614 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:06.111017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c6f60-0c44-4ee1-ae86-cb304cc21614 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:06.111017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c6f60-0c44-4ee1-ae86-cb304cc21614 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:06.112345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 666/1318] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:06.118099 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f02737-bcc3-4a29-ba04-e4ca107d8987 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:06.118498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f02737-bcc3-4a29-ba04-e4ca107d8987 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:06.118659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f02737-bcc3-4a29-ba04-e4ca107d8987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:06.183378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-67f02737-bcc3-4a29-ba04-e4ca107d8987 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:06.184605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-67f02737-bcc3-4a29-ba04-e4ca107d8987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:06.279486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 653/1319] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:06.585456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c6f60-0c44-4ee1-ae86-cb304cc21614 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:06.587095 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-383c6f60-0c44-4ee1-ae86-cb304cc21614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:06.703506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c6f60-0c44-4ee1-ae86-cb304cc21614 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:06.795931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f67c0a-0d86-4b85-98ec-0f853579b241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:06.796567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f67c0a-0d86-4b85-98ec-0f853579b241 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:06.796858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f67c0a-0d86-4b85-98ec-0f853579b241 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:06.797125 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f67c0a-0d86-4b85-98ec-0f853579b241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:06.799279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a1f67c0a-0d86-4b85-98ec-0f853579b241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:06.814902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:06.933552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f67c0a-0d86-4b85-98ec-0f853579b241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:06.934887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f67c0a-0d86-4b85-98ec-0f853579b241 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:06.957936 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 667/1320] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:09.577116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f05db1-6633-4a3c-90e6-326f81372790 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:09.577717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f05db1-6633-4a3c-90e6-326f81372790 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:09.577717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f05db1-6633-4a3c-90e6-326f81372790 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:09.579028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 654/1321] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:09.584285 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0099d16-9658-4def-89ff-f1d5d2172955 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:09.584739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0099d16-9658-4def-89ff-f1d5d2172955 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:09.584867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0099d16-9658-4def-89ff-f1d5d2172955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:09.652291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c0099d16-9658-4def-89ff-f1d5d2172955 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:09.653501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c0099d16-9658-4def-89ff-f1d5d2172955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:09.720188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:09.761596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 668/1322] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:10.067658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416d0a-37b6-4364-85b6-d810cf39c949 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:10.068592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416d0a-37b6-4364-85b6-d810cf39c949 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:10.069126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416d0a-37b6-4364-85b6-d810cf39c949 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:10.070755 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 655/1323] 10.222.0.60 () {54 vars in 965 bytes} [Wed Aug 31 11:35:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:10.076850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b4447-6557-447f-9842-df19af1270ee None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:10.077144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b4447-6557-447f-9842-df19af1270ee None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:10.077423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b4447-6557-447f-9842-df19af1270ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:10.162553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e45b4447-6557-447f-9842-df19af1270ee None None] MFA Rules not processed for user `30546ba7f5614f878a9ba322c7c55ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:10.163705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e45b4447-6557-447f-9842-df19af1270ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:10.261228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 669/1324] 10.222.0.60 () {58 vars in 1062 bytes} [Wed Aug 31 11:35:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:35:10.270521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc416d0a-37b6-4364-85b6-d810cf39c949 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:10.272131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cc416d0a-37b6-4364-85b6-d810cf39c949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:10.293734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:10.380067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc416d0a-37b6-4364-85b6-d810cf39c949 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:10.452445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53455720-2d57-4365-9197-f9f08ed6958e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:10.452994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53455720-2d57-4365-9197-f9f08ed6958e service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:10.453274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53455720-2d57-4365-9197-f9f08ed6958e service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:10.453544 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53455720-2d57-4365-9197-f9f08ed6958e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:10.455167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-53455720-2d57-4365-9197-f9f08ed6958e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:10.526586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:10.565580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53455720-2d57-4365-9197-f9f08ed6958e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:10.567022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53455720-2d57-4365-9197-f9f08ed6958e service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:10.585970 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 656/1325] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:12.763676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb5425-6557-4057-a0e1-d76627905582 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:12.764234 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb5425-6557-4057-a0e1-d76627905582 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:12.764357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb5425-6557-4057-a0e1-d76627905582 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:12.765762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 670/1326] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:12.770901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ee29f-a752-43d1-a512-50b352bbd14a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:12.772453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ee29f-a752-43d1-a512-50b352bbd14a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:12.772453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ee29f-a752-43d1-a512-50b352bbd14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:12.842862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-450ee29f-a752-43d1-a512-50b352bbd14a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:12.844076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-450ee29f-a752-43d1-a512-50b352bbd14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:12.954135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 657/1327] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:13.238896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb5425-6557-4057-a0e1-d76627905582 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:13.240793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a0fb5425-6557-4057-a0e1-d76627905582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:13.302037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:13.346623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb5425-6557-4057-a0e1-d76627905582 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:13.422774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09da8ee-ad12-4193-b38c-7dbe52dcea3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:13.423357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da8ee-ad12-4193-b38c-7dbe52dcea3b service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:13.423636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da8ee-ad12-4193-b38c-7dbe52dcea3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:13.423904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da8ee-ad12-4193-b38c-7dbe52dcea3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:13.425534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e09da8ee-ad12-4193-b38c-7dbe52dcea3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:13.533262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:13.535399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09da8ee-ad12-4193-b38c-7dbe52dcea3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:13.537533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09da8ee-ad12-4193-b38c-7dbe52dcea3b service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:13.558658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 671/1328] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:15.641979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265bb30-b1d9-43be-bb78-245dcb668563 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:15.642490 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265bb30-b1d9-43be-bb78-245dcb668563 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:15.642594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265bb30-b1d9-43be-bb78-245dcb668563 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:15.643880 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 658/1329] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:15.649000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558b991-7ed3-4094-a082-a5bee4202503 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:15.649399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558b991-7ed3-4094-a082-a5bee4202503 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:15.649681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558b991-7ed3-4094-a082-a5bee4202503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:15.752259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8558b991-7ed3-4094-a082-a5bee4202503 None None] MFA Rules not processed for user `8e67b785954f4fd4b095552017407c73`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:15.753967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8558b991-7ed3-4094-a082-a5bee4202503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:15.856905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 672/1330] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:15 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:16.140934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265bb30-b1d9-43be-bb78-245dcb668563 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:16.142554 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c265bb30-b1d9-43be-bb78-245dcb668563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:16.175468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:16.252899 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265bb30-b1d9-43be-bb78-245dcb668563 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:16.335025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a604fd-3720-4151-86f7-cd25d8b4bbfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:16.335576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a604fd-3720-4151-86f7-cd25d8b4bbfe service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:16.335859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a604fd-3720-4151-86f7-cd25d8b4bbfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:16.336217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a604fd-3720-4151-86f7-cd25d8b4bbfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:16.337808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c3a604fd-3720-4151-86f7-cd25d8b4bbfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:16.420521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:16.454847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a604fd-3720-4151-86f7-cd25d8b4bbfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:16.456295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a604fd-3720-4151-86f7-cd25d8b4bbfe service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:16.476922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 659/1331] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:18.904521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ce920-12ec-42ef-b71a-0e74ad5ef2be None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:18.905075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ce920-12ec-42ef-b71a-0e74ad5ef2be None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:18.905075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ce920-12ec-42ef-b71a-0e74ad5ef2be None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:18.906401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 673/1332] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:18.911474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bded7-1299-479d-846a-62e2ab35264f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:18.911944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bded7-1299-479d-846a-62e2ab35264f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:18.912056 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bded7-1299-479d-846a-62e2ab35264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:18.982317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3c9bded7-1299-479d-846a-62e2ab35264f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:18.983441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3c9bded7-1299-479d-846a-62e2ab35264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:19.080315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 660/1333] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:19.366085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ce920-12ec-42ef-b71a-0e74ad5ef2be None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:19.367620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf1ce920-12ec-42ef-b71a-0e74ad5ef2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:19.436113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:19.471223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ce920-12ec-42ef-b71a-0e74ad5ef2be None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:19.544058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed3bff-a484-4047-b97d-a720646b694e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:19.544680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3bff-a484-4047-b97d-a720646b694e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:19.544959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3bff-a484-4047-b97d-a720646b694e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:19.545230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3bff-a484-4047-b97d-a720646b694e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:19.546793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b9ed3bff-a484-4047-b97d-a720646b694e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:19.654965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed3bff-a484-4047-b97d-a720646b694e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:19.656277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed3bff-a484-4047-b97d-a720646b694e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:19.666688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:19.676016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 674/1334] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:21.642525 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0000be7-e847-497e-8e58-61dae33d8448 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:21.643096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0000be7-e847-497e-8e58-61dae33d8448 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:21.643096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0000be7-e847-497e-8e58-61dae33d8448 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:21.644408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 661/1335] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:21.650085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c173d-3fab-43a1-b1e5-21a2e2051d95 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:21.650503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c173d-3fab-43a1-b1e5-21a2e2051d95 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:21.650612 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c173d-3fab-43a1-b1e5-21a2e2051d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:21.719757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a14c173d-3fab-43a1-b1e5-21a2e2051d95 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:21.721144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a14c173d-3fab-43a1-b1e5-21a2e2051d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:21.816265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 675/1336] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:22.107357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0000be7-e847-497e-8e58-61dae33d8448 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:22.108922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d0000be7-e847-497e-8e58-61dae33d8448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:22.148835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:22.220907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0000be7-e847-497e-8e58-61dae33d8448 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:22.297812 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a14d37-c608-4572-8134-3e613a9acf1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:22.298413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a14d37-c608-4572-8134-3e613a9acf1a service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:22.298697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a14d37-c608-4572-8134-3e613a9acf1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:22.298978 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a14d37-c608-4572-8134-3e613a9acf1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:22.300617 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d2a14d37-c608-4572-8134-3e613a9acf1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:22.387080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:22.415730 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a14d37-c608-4572-8134-3e613a9acf1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:22.417206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a14d37-c608-4572-8134-3e613a9acf1a service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:22.434800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 662/1337] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:24.694612 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7178be-f288-470e-8131-205251bfee6f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:24.695081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7178be-f288-470e-8131-205251bfee6f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:24.695202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7178be-f288-470e-8131-205251bfee6f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:24.696798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 676/1338] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:24.701966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ab6b7-a6c2-4ce6-bca3-0847b5a82612 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:24.702386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ab6b7-a6c2-4ce6-bca3-0847b5a82612 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:24.702522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ab6b7-a6c2-4ce6-bca3-0847b5a82612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:24.771920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c79ab6b7-a6c2-4ce6-bca3-0847b5a82612 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:24.773088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c79ab6b7-a6c2-4ce6-bca3-0847b5a82612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:24.869119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 663/1339] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:25.150708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7178be-f288-470e-8131-205251bfee6f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:25.152259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9b7178be-f288-470e-8131-205251bfee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:25.233701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:25.262157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7178be-f288-470e-8131-205251bfee6f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:25.336799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5596d-0cc9-4505-9e51-8734879a1fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:25.337403 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5596d-0cc9-4505-9e51-8734879a1fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:25.337683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5596d-0cc9-4505-9e51-8734879a1fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:25.337956 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5596d-0cc9-4505-9e51-8734879a1fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:25.339538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1df5596d-0cc9-4505-9e51-8734879a1fe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:25.452026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5596d-0cc9-4505-9e51-8734879a1fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:25.453420 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5596d-0cc9-4505-9e51-8734879a1fe2 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:25.472636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 677/1340] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:29.605440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3095a9a-9baa-4cca-9245-54443df64679 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:29.606008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3095a9a-9baa-4cca-9245-54443df64679 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:29.606008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3095a9a-9baa-4cca-9245-54443df64679 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:29.607352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 664/1341] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:29.612638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380a0ad-2416-4811-b683-7e269a70b641 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:29.613033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380a0ad-2416-4811-b683-7e269a70b641 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:29.613209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380a0ad-2416-4811-b683-7e269a70b641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:29.618779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:29.691863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7380a0ad-2416-4811-b683-7e269a70b641 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:29.694366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7380a0ad-2416-4811-b683-7e269a70b641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:29.794550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 678/1342] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:30.086683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3095a9a-9baa-4cca-9245-54443df64679 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:30.088225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f3095a9a-9baa-4cca-9245-54443df64679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:30.138418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:30.197108 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3095a9a-9baa-4cca-9245-54443df64679 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:30.271656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281ec0f-e755-4657-b7c9-086082f18d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:30.272570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281ec0f-e755-4657-b7c9-086082f18d14 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:30.272852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281ec0f-e755-4657-b7c9-086082f18d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:30.273124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281ec0f-e755-4657-b7c9-086082f18d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:30.274755 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b281ec0f-e755-4657-b7c9-086082f18d14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:30.367889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:30.384577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281ec0f-e755-4657-b7c9-086082f18d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:30.386028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281ec0f-e755-4657-b7c9-086082f18d14 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:30.404597 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 665/1343] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:33.007082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32ddbc-68a4-4d09-8cf7-49be7493e3f4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:33.007588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32ddbc-68a4-4d09-8cf7-49be7493e3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:33.007588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32ddbc-68a4-4d09-8cf7-49be7493e3f4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:33.008907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 679/1344] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:35:33.014242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f302c4-0a81-4d42-96ad-317c0dc4a060 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:33.014748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f302c4-0a81-4d42-96ad-317c0dc4a060 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:33.014851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f302c4-0a81-4d42-96ad-317c0dc4a060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:33.081081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-26f302c4-0a81-4d42-96ad-317c0dc4a060 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:33.082288 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-26f302c4-0a81-4d42-96ad-317c0dc4a060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:33.180588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 666/1345] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:33.475423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32ddbc-68a4-4d09-8cf7-49be7493e3f4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:33.477034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6c32ddbc-68a4-4d09-8cf7-49be7493e3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:33.570339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:33.593882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32ddbc-68a4-4d09-8cf7-49be7493e3f4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:33.670566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82500c56-ac0d-4902-a762-b33a3a3e0f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:33.671168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82500c56-ac0d-4902-a762-b33a3a3e0f66 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:33.671456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82500c56-ac0d-4902-a762-b33a3a3e0f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:33.671727 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82500c56-ac0d-4902-a762-b33a3a3e0f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:33.673342 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-82500c56-ac0d-4902-a762-b33a3a3e0f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:33.791323 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82500c56-ac0d-4902-a762-b33a3a3e0f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:33.792881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82500c56-ac0d-4902-a762-b33a3a3e0f66 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:33.811852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 680/1346] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:37.772586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4af45-4f8f-4698-9843-032d398b6cdc None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:37.773149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4af45-4f8f-4698-9843-032d398b6cdc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:37.773149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4af45-4f8f-4698-9843-032d398b6cdc None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:37.774480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 667/1347] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:37.779946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e77fe-dbca-4d57-b270-352c37532d7a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:37.780393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e77fe-dbca-4d57-b270-352c37532d7a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:37.780528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e77fe-dbca-4d57-b270-352c37532d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:37.797214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:37.874410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-de8e77fe-dbca-4d57-b270-352c37532d7a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:37.875627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-de8e77fe-dbca-4d57-b270-352c37532d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:38.011529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 681/1348] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:38.299847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4af45-4f8f-4698-9843-032d398b6cdc None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:38.301681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-60a4af45-4f8f-4698-9843-032d398b6cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:38.366686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:38.418046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4af45-4f8f-4698-9843-032d398b6cdc None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:38.498644 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37610bdf-8d61-4fec-acb5-36ae5127618a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:38.499248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37610bdf-8d61-4fec-acb5-36ae5127618a service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:38.499526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37610bdf-8d61-4fec-acb5-36ae5127618a service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:38.499793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37610bdf-8d61-4fec-acb5-36ae5127618a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:38.501412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-37610bdf-8d61-4fec-acb5-36ae5127618a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:38.616128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:38.620794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37610bdf-8d61-4fec-acb5-36ae5127618a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:38.622273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37610bdf-8d61-4fec-acb5-36ae5127618a service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:38.642393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 668/1349] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:43.583412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbb3a7-73e5-401e-93d6-d4665061e909 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:43.585400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbb3a7-73e5-401e-93d6-d4665061e909 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:43.585908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbb3a7-73e5-401e-93d6-d4665061e909 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:43.588684 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 682/1350] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:35:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:43.594623 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edea28-216c-4cb8-9364-388eb1193a20 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:43.595571 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edea28-216c-4cb8-9364-388eb1193a20 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:43.595853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edea28-216c-4cb8-9364-388eb1193a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:43.691706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a0edea28-216c-4cb8-9364-388eb1193a20 None None] MFA Rules not processed for user `f9114a71a3a247188b263d87b6cb12c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:43.694126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a0edea28-216c-4cb8-9364-388eb1193a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:43.902890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 669/1351] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:35:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:48.008904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=59 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:48.148676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbc61a-8937-4a3e-802c-b49ff793a776 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:48.149266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbc61a-8937-4a3e-802c-b49ff793a776 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:48.149749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbc61a-8937-4a3e-802c-b49ff793a776 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:48.151523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 683/1352] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:48.157198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fb0fc-2241-484f-8550-955d5f206b62 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:48.157674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fb0fc-2241-484f-8550-955d5f206b62 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:48.157940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fb0fc-2241-484f-8550-955d5f206b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:48.213515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:48.238895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8a4fb0fc-2241-484f-8550-955d5f206b62 None None] MFA Rules not processed for user `f9114a71a3a247188b263d87b6cb12c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:48.240467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8a4fb0fc-2241-484f-8550-955d5f206b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:48.357441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 670/1353] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:35:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:49.796438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19bb4d-24f2-43bf-8f3a-c8317985677d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:49.797010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19bb4d-24f2-43bf-8f3a-c8317985677d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:49.797010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19bb4d-24f2-43bf-8f3a-c8317985677d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:49.798350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 684/1354] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:49.803345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5b1ae-8d4d-483f-a085-9716b62b127f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:49.803754 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5b1ae-8d4d-483f-a085-9716b62b127f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:49.804025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5b1ae-8d4d-483f-a085-9716b62b127f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:49.881778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e3c5b1ae-8d4d-483f-a085-9716b62b127f None None] MFA Rules not processed for user `f9114a71a3a247188b263d87b6cb12c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:49.882986 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e3c5b1ae-8d4d-483f-a085-9716b62b127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:49.928370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:50.003068 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 671/1355] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:35:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:35:53.165252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3389b5-b57e-4248-8281-808cfb437b69 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:53.165784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3389b5-b57e-4248-8281-808cfb437b69 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:53.165784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3389b5-b57e-4248-8281-808cfb437b69 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:53.167119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 685/1356] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:53.172417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac9972-52ce-4e96-87e5-451e4bfd836b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:53.172892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac9972-52ce-4e96-87e5-451e4bfd836b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:53.173016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac9972-52ce-4e96-87e5-451e4bfd836b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:53.244519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b3ac9972-52ce-4e96-87e5-451e4bfd836b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:53.244519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b3ac9972-52ce-4e96-87e5-451e4bfd836b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:53.386627 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 672/1357] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:53.719408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861136ed-b944-4d60-98e8-e7cf326e1a96 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:53.720013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861136ed-b944-4d60-98e8-e7cf326e1a96 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:53.720549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861136ed-b944-4d60-98e8-e7cf326e1a96 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:53.722498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 686/1358] 10.222.0.60 () {54 vars in 962 bytes} [Wed Aug 31 11:35:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:53.733871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709021b-ebfd-434a-bb03-786e6bd8013d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:53.734979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709021b-ebfd-434a-bb03-786e6bd8013d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:53.735585 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709021b-ebfd-434a-bb03-786e6bd8013d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:53.754693 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:53.821655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c709021b-ebfd-434a-bb03-786e6bd8013d None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:53.823018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c709021b-ebfd-434a-bb03-786e6bd8013d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:53.961596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 673/1359] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Aug 31 11:35:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:35:53.969923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861136ed-b944-4d60-98e8-e7cf326e1a96 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:53.971632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-861136ed-b944-4d60-98e8-e7cf326e1a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:54.137190 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861136ed-b944-4d60-98e8-e7cf326e1a96 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:54.219265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:54.263082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7d08b-a936-4b84-80f5-3ca8991a569d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:54.263839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7d08b-a936-4b84-80f5-3ca8991a569d service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:54.264222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7d08b-a936-4b84-80f5-3ca8991a569d service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:54.264773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7d08b-a936-4b84-80f5-3ca8991a569d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:54.266716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dfd7d08b-a936-4b84-80f5-3ca8991a569d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:54.363416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7d08b-a936-4b84-80f5-3ca8991a569d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:54.365521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7d08b-a936-4b84-80f5-3ca8991a569d service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:54.386710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 687/1360] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:35:56.542869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021f8eb-fc2f-45c8-bf5c-352a6037370e None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:56.543404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021f8eb-fc2f-45c8-bf5c-352a6037370e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:56.543404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021f8eb-fc2f-45c8-bf5c-352a6037370e None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:56.544885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 674/1361] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:35:56.549908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f36a6b-b2a9-4a3a-8eaa-b79f852d9de9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:56.550331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f36a6b-b2a9-4a3a-8eaa-b79f852d9de9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:56.550465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f36a6b-b2a9-4a3a-8eaa-b79f852d9de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:56.621329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-39f36a6b-b2a9-4a3a-8eaa-b79f852d9de9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:56.622523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-39f36a6b-b2a9-4a3a-8eaa-b79f852d9de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:56.662728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:56.735769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 688/1362] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:35:57.031699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021f8eb-fc2f-45c8-bf5c-352a6037370e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:57.033335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8021f8eb-fc2f-45c8-bf5c-352a6037370e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:57.116107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:57.205874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021f8eb-fc2f-45c8-bf5c-352a6037370e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:57.330608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e22c82-1e39-4328-92e9-240db8cbc0c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:57.331225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e22c82-1e39-4328-92e9-240db8cbc0c0 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:57.331506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e22c82-1e39-4328-92e9-240db8cbc0c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:57.331781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e22c82-1e39-4328-92e9-240db8cbc0c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:57.333647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-91e22c82-1e39-4328-92e9-240db8cbc0c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:57.358009 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:57.438450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e22c82-1e39-4328-92e9-240db8cbc0c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:57.440158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e22c82-1e39-4328-92e9-240db8cbc0c0 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:57.459725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 675/1363] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:59.695553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f9524-661d-4d1a-9fa3-859ff08f9e18 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:59.696059 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f9524-661d-4d1a-9fa3-859ff08f9e18 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:59.696395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f9524-661d-4d1a-9fa3-859ff08f9e18 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:59.699617 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 689/1364] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:35:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:59.703816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa09568-4dcc-4776-ad74-7073ece36d06 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:59.704264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa09568-4dcc-4776-ad74-7073ece36d06 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:59.704366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa09568-4dcc-4776-ad74-7073ece36d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:59.781142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2fa09568-4dcc-4776-ad74-7073ece36d06 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:59.782399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2fa09568-4dcc-4776-ad74-7073ece36d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:59.886187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:59.892694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 676/1365] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:00.185396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d46b4f-43af-4231-862c-85c81555ab7e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:00.186065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d46b4f-43af-4231-862c-85c81555ab7e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:00.187462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d46b4f-43af-4231-862c-85c81555ab7e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:00.189059 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 690/1366] 10.222.0.60 () {54 vars in 962 bytes} [Wed Aug 31 11:36:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:00.196795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660dbd6-bb5f-4a11-8a8f-9f4d31dceccb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:00.197742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660dbd6-bb5f-4a11-8a8f-9f4d31dceccb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:00.198164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660dbd6-bb5f-4a11-8a8f-9f4d31dceccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:00.281497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a660dbd6-bb5f-4a11-8a8f-9f4d31dceccb None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:00.282819 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a660dbd6-bb5f-4a11-8a8f-9f4d31dceccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:00.415565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 677/1367] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Aug 31 11:36:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:00.423729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d46b4f-43af-4231-862c-85c81555ab7e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:00.425384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f9d46b4f-43af-4231-862c-85c81555ab7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:00.579416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d46b4f-43af-4231-862c-85c81555ab7e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:00.602299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:00.711182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6ee6a-bdbc-422d-9146-2b7a670d3ae7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:00.711959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6ee6a-bdbc-422d-9146-2b7a670d3ae7 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:00.712186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6ee6a-bdbc-422d-9146-2b7a670d3ae7 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:00.712459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6ee6a-bdbc-422d-9146-2b7a670d3ae7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:00.714075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e3d6ee6a-bdbc-422d-9146-2b7a670d3ae7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:00.810098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6ee6a-bdbc-422d-9146-2b7a670d3ae7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:00.811697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6ee6a-bdbc-422d-9146-2b7a670d3ae7 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:00.832048 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 691/1368] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:01.513216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8173e2-8674-41dc-8f66-f1a7ff77775e None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.514378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8173e2-8674-41dc-8f66-f1a7ff77775e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.514870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8173e2-8674-41dc-8f66-f1a7ff77775e None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.516831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 678/1369] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:36:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:01.522787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7678f-427c-4252-8059-52c48b7d1c57 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.523092 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7678f-427c-4252-8059-52c48b7d1c57 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.523365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7678f-427c-4252-8059-52c48b7d1c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.558702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:01.597452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c7d7678f-427c-4252-8059-52c48b7d1c57 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:01.599820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c7d7678f-427c-4252-8059-52c48b7d1c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:01.714732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 692/1370] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:02.318196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3850f-300f-4de0-afed-4406be110118 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:02.318616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3850f-300f-4de0-afed-4406be110118 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:02.319014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3850f-300f-4de0-afed-4406be110118 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:02.320908 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 679/1371] 10.222.0.60 () {54 vars in 964 bytes} [Wed Aug 31 11:36:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:02.327097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c9ea9-4626-4d70-953f-3335380d2800 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:02.327548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c9ea9-4626-4d70-953f-3335380d2800 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:02.328122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c9ea9-4626-4d70-953f-3335380d2800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:02.418105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-db0c9ea9-4626-4d70-953f-3335380d2800 None None] MFA Rules not processed for user `e8dec99551ff4c31a17247cde3515acd`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:02.419877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-db0c9ea9-4626-4d70-953f-3335380d2800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:02.476061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:02.512232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 693/1372] 10.222.0.60 () {58 vars in 1061 bytes} [Wed Aug 31 11:36:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:02.519745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3850f-300f-4de0-afed-4406be110118 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:02.522588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d2e3850f-300f-4de0-afed-4406be110118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:02.585773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:02.646261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3850f-300f-4de0-afed-4406be110118 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:02.762410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a77cb8-5987-49d3-884c-c068bbb5e3ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:02.763919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a77cb8-5987-49d3-884c-c068bbb5e3ce service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:02.766584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a77cb8-5987-49d3-884c-c068bbb5e3ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:02.767317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a77cb8-5987-49d3-884c-c068bbb5e3ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:02.771386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b8a77cb8-5987-49d3-884c-c068bbb5e3ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:02.806654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866b0fe-238e-49b9-a1a5-67cf476a693e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:02.807090 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866b0fe-238e-49b9-a1a5-67cf476a693e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:02.807402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866b0fe-238e-49b9-a1a5-67cf476a693e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:02.810298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 694/1373] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:36:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:02.814691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81eda4-c3b1-471e-853b-f8051d4bd68b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:02.815004 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81eda4-c3b1-471e-853b-f8051d4bd68b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:02.815152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81eda4-c3b1-471e-853b-f8051d4bd68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:02.893378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3a81eda4-c3b1-471e-853b-f8051d4bd68b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:02.894057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3a81eda4-c3b1-471e-853b-f8051d4bd68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:02.920286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:02.933796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a77cb8-5987-49d3-884c-c068bbb5e3ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:02.935320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a77cb8-5987-49d3-884c-c068bbb5e3ce service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:02.956453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 680/1374] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:36:03.012780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 695/1375] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:36:03.316543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a77cb8-5987-49d3-884c-c068bbb5e3ce service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:03.318540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b8a77cb8-5987-49d3-884c-c068bbb5e3ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:03.422856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a77cb8-5987-49d3-884c-c068bbb5e3ce service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:03.527700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:03.541776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465536ac-ab61-48f8-a8a4-092b3ebeda98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:03.542386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465536ac-ab61-48f8-a8a4-092b3ebeda98 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:03.542674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465536ac-ab61-48f8-a8a4-092b3ebeda98 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:03.542951 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465536ac-ab61-48f8-a8a4-092b3ebeda98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:03.544624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-465536ac-ab61-48f8-a8a4-092b3ebeda98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:03.638332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465536ac-ab61-48f8-a8a4-092b3ebeda98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:03.639953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465536ac-ab61-48f8-a8a4-092b3ebeda98 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:03.659382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 681/1376] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:05.637199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fda12e-a625-46db-8469-beea09b7f2b1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:05.637681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fda12e-a625-46db-8469-beea09b7f2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:05.637681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fda12e-a625-46db-8469-beea09b7f2b1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:05.639132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 696/1377] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:05.644442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce801cd2-b188-4a19-a41f-ca158049068a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:05.644937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce801cd2-b188-4a19-a41f-ca158049068a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:05.645165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce801cd2-b188-4a19-a41f-ca158049068a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:05.715481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ce801cd2-b188-4a19-a41f-ca158049068a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:05.716733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ce801cd2-b188-4a19-a41f-ca158049068a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:05.775468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:05.819783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 682/1378] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:06.110324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fda12e-a625-46db-8469-beea09b7f2b1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:06.111834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d6fda12e-a625-46db-8469-beea09b7f2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:06.165683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:06.260857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fda12e-a625-46db-8469-beea09b7f2b1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:06.374474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6569914-2f9e-4e12-89d9-3739146b6140 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:06.375331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6569914-2f9e-4e12-89d9-3739146b6140 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:06.375762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6569914-2f9e-4e12-89d9-3739146b6140 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:06.376180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6569914-2f9e-4e12-89d9-3739146b6140 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:06.378094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b6569914-2f9e-4e12-89d9-3739146b6140 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:06.391058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:06.503245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6569914-2f9e-4e12-89d9-3739146b6140 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:06.509920 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6569914-2f9e-4e12-89d9-3739146b6140 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:06.528209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 697/1379] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:08.540673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974dc37-ab8c-4636-8a5f-fb2d4c3c4765 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:08.541222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974dc37-ab8c-4636-8a5f-fb2d4c3c4765 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:08.541222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974dc37-ab8c-4636-8a5f-fb2d4c3c4765 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:08.542969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 683/1380] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:08.549042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e379b-2de1-472e-af8a-640fd1d322a8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:08.549598 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e379b-2de1-472e-af8a-640fd1d322a8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:08.549598 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e379b-2de1-472e-af8a-640fd1d322a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:08.626678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-948e379b-2de1-472e-af8a-640fd1d322a8 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:08.627873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-948e379b-2de1-472e-af8a-640fd1d322a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:08.732036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:08.745957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 698/1381] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:09.033518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974dc37-ab8c-4636-8a5f-fb2d4c3c4765 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:09.035041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8974dc37-ab8c-4636-8a5f-fb2d4c3c4765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:09.195194 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974dc37-ab8c-4636-8a5f-fb2d4c3c4765 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:09.247402 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:09.331990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82434581-de34-4174-842d-e499f6f04cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:09.332724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434581-de34-4174-842d-e499f6f04cfb service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:09.333020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434581-de34-4174-842d-e499f6f04cfb service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:09.333297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434581-de34-4174-842d-e499f6f04cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:09.335039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-82434581-de34-4174-842d-e499f6f04cfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:09.458471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82434581-de34-4174-842d-e499f6f04cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:09.460047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82434581-de34-4174-842d-e499f6f04cfb service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:09.480674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 684/1382] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:11.430214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240956e1-a089-4ee8-bc63-f259fe03ef4f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:11.430798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240956e1-a089-4ee8-bc63-f259fe03ef4f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:11.430798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240956e1-a089-4ee8-bc63-f259fe03ef4f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:11.432202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 699/1383] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:11.437889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f82608-8027-4a64-bb28-4bcd68a1db5c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:11.438305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f82608-8027-4a64-bb28-4bcd68a1db5c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:11.438469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f82608-8027-4a64-bb28-4bcd68a1db5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:11.512109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-88f82608-8027-4a64-bb28-4bcd68a1db5c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:11.513406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-88f82608-8027-4a64-bb28-4bcd68a1db5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:11.515545 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:11.624097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 685/1384] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:36:11.915659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240956e1-a089-4ee8-bc63-f259fe03ef4f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:11.918214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-240956e1-a089-4ee8-bc63-f259fe03ef4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:12.089848 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240956e1-a089-4ee8-bc63-f259fe03ef4f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:12.165680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:12.228151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31abe5-2a2f-4d65-bc55-5d889b5347e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:12.228996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31abe5-2a2f-4d65-bc55-5d889b5347e6 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:12.229278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31abe5-2a2f-4d65-bc55-5d889b5347e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:12.229892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31abe5-2a2f-4d65-bc55-5d889b5347e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:12.232162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dd31abe5-2a2f-4d65-bc55-5d889b5347e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:12.332343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31abe5-2a2f-4d65-bc55-5d889b5347e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:12.334626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31abe5-2a2f-4d65-bc55-5d889b5347e6 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:12.354024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 700/1385] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:36:14.440919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b832a31-289c-48dc-b5d7-337d0ae74bd8 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:14.441475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b832a31-289c-48dc-b5d7-337d0ae74bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:14.441475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b832a31-289c-48dc-b5d7-337d0ae74bd8 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:14.444090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 686/1386] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:36:14.450561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9edcc-8fda-4b53-8175-0705e27b8eaf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:14.451042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9edcc-8fda-4b53-8175-0705e27b8eaf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:14.451161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9edcc-8fda-4b53-8175-0705e27b8eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:14.522274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-18d9edcc-8fda-4b53-8175-0705e27b8eaf None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:14.523500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-18d9edcc-8fda-4b53-8175-0705e27b8eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:14.551058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:14.625461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 701/1387] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:36:14.925630 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b832a31-289c-48dc-b5d7-337d0ae74bd8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:14.928618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5b832a31-289c-48dc-b5d7-337d0ae74bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:15.071241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:15.090121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b832a31-289c-48dc-b5d7-337d0ae74bd8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:15.207018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ffedd-755f-43f2-8d51-11d6391f81b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:15.208152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ffedd-755f-43f2-8d51-11d6391f81b7 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:15.208446 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ffedd-755f-43f2-8d51-11d6391f81b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:15.208763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ffedd-755f-43f2-8d51-11d6391f81b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:15.210467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0a2ffedd-755f-43f2-8d51-11d6391f81b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:15.318855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ffedd-755f-43f2-8d51-11d6391f81b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:15.320519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ffedd-755f-43f2-8d51-11d6391f81b7 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:15.325915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:15.340430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 687/1388] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:17.446712 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e5147-8fc9-439b-9e94-1249ad5c51ae None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:17.447250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e5147-8fc9-439b-9e94-1249ad5c51ae None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:17.447250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e5147-8fc9-439b-9e94-1249ad5c51ae None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:17.448620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 702/1389] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:17.453924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624189d-7f7c-4d09-adb9-4bea3bfa8b77 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:17.454349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624189d-7f7c-4d09-adb9-4bea3bfa8b77 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:17.454471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624189d-7f7c-4d09-adb9-4bea3bfa8b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:17.524397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c624189d-7f7c-4d09-adb9-4bea3bfa8b77 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:17.525645 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c624189d-7f7c-4d09-adb9-4bea3bfa8b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:17.625287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 688/1390] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:17.912385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e5147-8fc9-439b-9e94-1249ad5c51ae None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:17.913956 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-863e5147-8fc9-439b-9e94-1249ad5c51ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:18.063227 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e5147-8fc9-439b-9e94-1249ad5c51ae None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:18.073298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:18.180225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91853c3-5a85-4516-a0b9-d5834b931ec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:18.180918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91853c3-5a85-4516-a0b9-d5834b931ec5 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:18.181361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91853c3-5a85-4516-a0b9-d5834b931ec5 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:18.181361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91853c3-5a85-4516-a0b9-d5834b931ec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:18.181763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd063c-3461-4be2-b4b8-d6916bba6bcb None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:18.182040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd063c-3461-4be2-b4b8-d6916bba6bcb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:18.182316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd063c-3461-4be2-b4b8-d6916bba6bcb None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:18.183087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b91853c3-5a85-4516-a0b9-d5834b931ec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:18.183852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 689/1391] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:18.188587 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5160-7f0d-43d6-999d-454345da27dd None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:18.188871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5160-7f0d-43d6-999d-454345da27dd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:18.189146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5160-7f0d-43d6-999d-454345da27dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:18.257487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-32de5160-7f0d-43d6-999d-454345da27dd None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:18.258703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-32de5160-7f0d-43d6-999d-454345da27dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:18.278179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91853c3-5a85-4516-a0b9-d5834b931ec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:18.279790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91853c3-5a85-4516-a0b9-d5834b931ec5 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:18.279939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:18.299895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 703/1392] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:36:18.364647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 690/1393] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:36:18.667883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36c81a-a3b1-4313-860b-d7e0fee9a041 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:18.668379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36c81a-a3b1-4313-860b-d7e0fee9a041 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:18.668509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36c81a-a3b1-4313-860b-d7e0fee9a041 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:18.669773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 704/1394] 10.222.0.60 () {54 vars in 967 bytes} [Wed Aug 31 11:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:18.674821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f792e5c-9fef-40f2-8745-63788a9747fb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:18.675224 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f792e5c-9fef-40f2-8745-63788a9747fb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:18.675441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f792e5c-9fef-40f2-8745-63788a9747fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:18.752296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3f792e5c-9fef-40f2-8745-63788a9747fb None None] MFA Rules not processed for user `18ec048788904bc58245e878a8644091`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:18.753466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3f792e5c-9fef-40f2-8745-63788a9747fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:18.830094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 691/1395] 10.222.0.60 () {58 vars in 1064 bytes} [Wed Aug 31 11:36:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:18.838114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36c81a-a3b1-4313-860b-d7e0fee9a041 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:18.839608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ae36c81a-a3b1-4313-860b-d7e0fee9a041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:18.854229 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:18.947461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36c81a-a3b1-4313-860b-d7e0fee9a041 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:19.022009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57ee7f-10dd-4761-be6e-090203a16746 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ec048788904bc58245e878a8644091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:19.022523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57ee7f-10dd-4761-be6e-090203a16746 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:19.022803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57ee7f-10dd-4761-be6e-090203a16746 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:19.023073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57ee7f-10dd-4761-be6e-090203a16746 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:19.024752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4c57ee7f-10dd-4761-be6e-090203a16746 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:19.099802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:19.142939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57ee7f-10dd-4761-be6e-090203a16746 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:19.144362 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57ee7f-10dd-4761-be6e-090203a16746 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:19.163268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 705/1396] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:19.917383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576200f-b7cb-4a5b-ab68-db411c6d679d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:19.917883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576200f-b7cb-4a5b-ab68-db411c6d679d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:19.917980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576200f-b7cb-4a5b-ab68-db411c6d679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:19.940311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:19.990394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6576200f-b7cb-4a5b-ab68-db411c6d679d None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:19.991581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6576200f-b7cb-4a5b-ab68-db411c6d679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:20.079308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 692/1397] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:36:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:20.121767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5e7c0-c7c7-451a-821a-4563c5dc7f28 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:20.122269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5e7c0-c7c7-451a-821a-4563c5dc7f28 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:20.122377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5e7c0-c7c7-451a-821a-4563c5dc7f28 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:20.123720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 706/1398] 10.222.0.60 () {54 vars in 943 bytes} [Wed Aug 31 11:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:20.130447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674cfb4-77eb-437d-8e68-e79eb29b16cb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:20.130807 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674cfb4-77eb-437d-8e68-e79eb29b16cb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:20.131150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674cfb4-77eb-437d-8e68-e79eb29b16cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:20.211657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8674cfb4-77eb-437d-8e68-e79eb29b16cb None None] MFA Rules not processed for user `3384c24dd081414bbf0a2e3d9f71126b`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:20.212970 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8674cfb4-77eb-437d-8e68-e79eb29b16cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:20.262802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:20.280327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87c007-c453-4e7c-8dbd-defaba8fbafa None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:20.280651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87c007-c453-4e7c-8dbd-defaba8fbafa None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:20.280931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87c007-c453-4e7c-8dbd-defaba8fbafa None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:20.282264 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 707/1399] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:36:20.288101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e85ca9-a4de-40e4-8920-8fab2ceddab5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:20.288383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e85ca9-a4de-40e4-8920-8fab2ceddab5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:20.288695 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e85ca9-a4de-40e4-8920-8fab2ceddab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:20.321848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 693/1400] 10.222.0.60 () {58 vars in 1040 bytes} [Wed Aug 31 11:36:20 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:36:20.331081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674cfb4-77eb-437d-8e68-e79eb29b16cb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:20.345613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674cfb4-77eb-437d-8e68-e79eb29b16cb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:20.360454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-21e85ca9-a4de-40e4-8920-8fab2ceddab5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:20.361726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-21e85ca9-a4de-40e4-8920-8fab2ceddab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:20.456407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5038f-29d5-4bf8-b091-a32eb457b877 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:20.457039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5038f-29d5-4bf8-b091-a32eb457b877 service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:20.457432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5038f-29d5-4bf8-b091-a32eb457b877 service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:20.458253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5038f-29d5-4bf8-b091-a32eb457b877 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:20.459887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-41b5038f-29d5-4bf8-b091-a32eb457b877 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:20.464644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 708/1401] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:36:20.513042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:20.544179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5038f-29d5-4bf8-b091-a32eb457b877 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:20.545830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5038f-29d5-4bf8-b091-a32eb457b877 service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:20.554359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 694/1402] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:36:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 11:36:20.760484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e85ca9-a4de-40e4-8920-8fab2ceddab5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:20.761843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-21e85ca9-a4de-40e4-8920-8fab2ceddab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:20.791356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:20.890962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e85ca9-a4de-40e4-8920-8fab2ceddab5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:21.028764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce61fd8-cfdb-4061-ab5e-2732b10e53f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:21.029547 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61fd8-cfdb-4061-ab5e-2732b10e53f4 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:21.030169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61fd8-cfdb-4061-ab5e-2732b10e53f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:21.030517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61fd8-cfdb-4061-ab5e-2732b10e53f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:21.032199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6ce61fd8-cfdb-4061-ab5e-2732b10e53f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:21.052419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:21.134402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce61fd8-cfdb-4061-ab5e-2732b10e53f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:21.136890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce61fd8-cfdb-4061-ab5e-2732b10e53f4 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:21.156215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 709/1403] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:23.105512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2c047-2388-4a87-adfe-8dfcb10331bb None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:23.106042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2c047-2388-4a87-adfe-8dfcb10331bb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:23.106042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2c047-2388-4a87-adfe-8dfcb10331bb None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:23.107398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 695/1404] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:23.112441 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39224a1a-c900-4d02-9177-7310800d7f3f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:23.112866 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39224a1a-c900-4d02-9177-7310800d7f3f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:23.113057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39224a1a-c900-4d02-9177-7310800d7f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:23.188449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-39224a1a-c900-4d02-9177-7310800d7f3f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:23.189676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-39224a1a-c900-4d02-9177-7310800d7f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:23.305938 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:23.316535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 710/1405] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:23.607421 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2c047-2388-4a87-adfe-8dfcb10331bb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:23.609036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b6a2c047-2388-4a87-adfe-8dfcb10331bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:23.783780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2c047-2388-4a87-adfe-8dfcb10331bb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:23.837868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:23.918539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeee0f3-e454-4dee-aec1-84324f0bdbd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:23.919152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeee0f3-e454-4dee-aec1-84324f0bdbd8 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:23.919734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeee0f3-e454-4dee-aec1-84324f0bdbd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:23.920163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeee0f3-e454-4dee-aec1-84324f0bdbd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:23.922258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bbeee0f3-e454-4dee-aec1-84324f0bdbd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:24.034369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeee0f3-e454-4dee-aec1-84324f0bdbd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:24.036709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeee0f3-e454-4dee-aec1-84324f0bdbd8 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:24.055696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 696/1406] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:26.140240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083db4-7ebb-4b94-90e6-848dc82b3dac None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:26.140875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083db4-7ebb-4b94-90e6-848dc82b3dac None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:26.140875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083db4-7ebb-4b94-90e6-848dc82b3dac None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:26.142335 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 711/1407] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:26.147644 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daebe81-5b39-4877-8f8f-4cf7707dce1c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:26.148084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daebe81-5b39-4877-8f8f-4cf7707dce1c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:26.148182 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daebe81-5b39-4877-8f8f-4cf7707dce1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:26.222447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6daebe81-5b39-4877-8f8f-4cf7707dce1c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:26.224688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6daebe81-5b39-4877-8f8f-4cf7707dce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:26.226283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:26.335425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 697/1408] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:26.638136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6083db4-7ebb-4b94-90e6-848dc82b3dac None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:26.639640 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a6083db4-7ebb-4b94-90e6-848dc82b3dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:26.813913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6083db4-7ebb-4b94-90e6-848dc82b3dac None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:26.887315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:26.941442 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361034f2-f916-49ae-9289-eefedcf15a82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:26.943102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361034f2-f916-49ae-9289-eefedcf15a82 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:26.943501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361034f2-f916-49ae-9289-eefedcf15a82 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:26.943870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361034f2-f916-49ae-9289-eefedcf15a82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:26.945635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-361034f2-f916-49ae-9289-eefedcf15a82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:27.044007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361034f2-f916-49ae-9289-eefedcf15a82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:27.045749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361034f2-f916-49ae-9289-eefedcf15a82 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:27.064933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 712/1409] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:36.332364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=42 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:37.062565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=64 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:42.806120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbf2ce-9bc8-43ef-bb41-babea29f9b94 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:42.806657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbf2ce-9bc8-43ef-bb41-babea29f9b94 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:42.806657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbf2ce-9bc8-43ef-bb41-babea29f9b94 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:42.808142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 698/1410] 10.222.0.60 () {54 vars in 918 bytes} [Wed Aug 31 11:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:42.813149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeeb8a-ad00-4a51-81fa-67add691f352 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:42.813550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeeb8a-ad00-4a51-81fa-67add691f352 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:42.813687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeeb8a-ad00-4a51-81fa-67add691f352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:42.888466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9ffeeb8a-ad00-4a51-81fa-67add691f352 None None] MFA Rules not processed for user `f9114a71a3a247188b263d87b6cb12c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:42.889681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9ffeeb8a-ad00-4a51-81fa-67add691f352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:42.996945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 713/1411] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Aug 31 11:36:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:43.011094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57531ed6-55b5-402b-a5cd-427e8cb867c9 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:43.011382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57531ed6-55b5-402b-a5cd-427e8cb867c9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:43.011661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57531ed6-55b5-402b-a5cd-427e8cb867c9 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:43.013084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 699/1412] 10.222.0.60 () {54 vars in 918 bytes} [Wed Aug 31 11:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:43.017512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbbb93-99c9-4dea-ac7b-6c9b84a6a57c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:43.017799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbbb93-99c9-4dea-ac7b-6c9b84a6a57c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:43.018069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbbb93-99c9-4dea-ac7b-6c9b84a6a57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:43.208957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:43.212277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dadbbb93-99c9-4dea-ac7b-6c9b84a6a57c None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:43.213987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dadbbb93-99c9-4dea-ac7b-6c9b84a6a57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:43.329775 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 714/1413] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Aug 31 11:36:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:36:45.721852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e50a2-1136-469f-8d20-0f251ecf2c0c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:45.722380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e50a2-1136-469f-8d20-0f251ecf2c0c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:45.722527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e50a2-1136-469f-8d20-0f251ecf2c0c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:45.723857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:45.728984 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a463be7-d528-4c34-b246-c0aaa5606a2b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:45.729413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a463be7-d528-4c34-b246-c0aaa5606a2b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:45.729521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a463be7-d528-4c34-b246-c0aaa5606a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:45.794310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1a463be7-d528-4c34-b246-c0aaa5606a2b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:45.795493 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a463be7-d528-4c34-b246-c0aaa5606a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:45.849798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:45.890726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:46.176340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e50a2-1136-469f-8d20-0f251ecf2c0c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:46.177911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-624e50a2-1136-469f-8d20-0f251ecf2c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:46.325530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e50a2-1136-469f-8d20-0f251ecf2c0c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:46.400011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:46.443020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7cb74-7242-487d-9f35-7907d7fe6546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:46.443623 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7cb74-7242-487d-9f35-7907d7fe6546 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:46.443902 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7cb74-7242-487d-9f35-7907d7fe6546 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:46.444171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7cb74-7242-487d-9f35-7907d7fe6546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:46.445810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4dc7cb74-7242-487d-9f35-7907d7fe6546 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:46.535410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7cb74-7242-487d-9f35-7907d7fe6546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:46.537035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7cb74-7242-487d-9f35-7907d7fe6546 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:46.554815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 701/1416] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:47.443792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0970-45b1-472f-ba1c-df707461c3cb None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:47.444368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0970-45b1-472f-ba1c-df707461c3cb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:47.444368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0970-45b1-472f-ba1c-df707461c3cb None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:47.445741 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:47.450798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7d5f6-3396-40e1-9d20-c87d44332e2d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:47.451222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7d5f6-3396-40e1-9d20-c87d44332e2d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:47.451340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7d5f6-3396-40e1-9d20-c87d44332e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:47.527859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-89e7d5f6-3396-40e1-9d20-c87d44332e2d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:47.529127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-89e7d5f6-3396-40e1-9d20-c87d44332e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:47.556381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:47.625707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:47.775186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb0970-45b1-472f-ba1c-df707461c3cb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:47.776641 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-08cb0970-45b1-472f-ba1c-df707461c3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:47.926616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb0970-45b1-472f-ba1c-df707461c3cb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:47.967754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:48.041948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7154861-9937-45f0-b2ce-a18a43819cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:48.042466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7154861-9937-45f0-b2ce-a18a43819cfb service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:48.042747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7154861-9937-45f0-b2ce-a18a43819cfb service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:48.043015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7154861-9937-45f0-b2ce-a18a43819cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:48.044634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c7154861-9937-45f0-b2ce-a18a43819cfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:48.132583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7154861-9937-45f0-b2ce-a18a43819cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:48.134157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7154861-9937-45f0-b2ce-a18a43819cfb service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:48.152393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 717/1419] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:49.056613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a76dcf-b9a9-4bc3-9544-e840a0d95a01 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:49.057142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a76dcf-b9a9-4bc3-9544-e840a0d95a01 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:49.057142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a76dcf-b9a9-4bc3-9544-e840a0d95a01 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:49.058482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:49.063455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637be239-c61a-4f10-9ca5-f44d813a39d6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:49.063864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637be239-c61a-4f10-9ca5-f44d813a39d6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:49.064006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637be239-c61a-4f10-9ca5-f44d813a39d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:49.120312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:49.142351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-637be239-c61a-4f10-9ca5-f44d813a39d6 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:49.143581 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-637be239-c61a-4f10-9ca5-f44d813a39d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:49.243565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:49.402086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a76dcf-b9a9-4bc3-9544-e840a0d95a01 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:49.403641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-02a76dcf-b9a9-4bc3-9544-e840a0d95a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:49.552173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a76dcf-b9a9-4bc3-9544-e840a0d95a01 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:49.564080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:49.664767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc694bf1-830a-44e1-8836-8c8f31ae037b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:49.665285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc694bf1-830a-44e1-8836-8c8f31ae037b service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:49.665562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc694bf1-830a-44e1-8836-8c8f31ae037b service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:49.665831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc694bf1-830a-44e1-8836-8c8f31ae037b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:49.667427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bc694bf1-830a-44e1-8836-8c8f31ae037b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:49.760923 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc694bf1-830a-44e1-8836-8c8f31ae037b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:49.762558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc694bf1-830a-44e1-8836-8c8f31ae037b service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:49.781040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 704/1422] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:50.642307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83bc51-d02b-4bac-8c78-7431380b917e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:50.642889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83bc51-d02b-4bac-8c78-7431380b917e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:50.642889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83bc51-d02b-4bac-8c78-7431380b917e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:50.644204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:50.649542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345f030-5001-4f0c-bab7-8d96e0a85f53 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:50.649988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345f030-5001-4f0c-bab7-8d96e0a85f53 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:50.650093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345f030-5001-4f0c-bab7-8d96e0a85f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:50.669519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:50.726843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a345f030-5001-4f0c-bab7-8d96e0a85f53 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:50.728054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a345f030-5001-4f0c-bab7-8d96e0a85f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:50.822943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:50.972782 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83bc51-d02b-4bac-8c78-7431380b917e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:50.974188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-db83bc51-d02b-4bac-8c78-7431380b917e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:51.083944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:51.119332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83bc51-d02b-4bac-8c78-7431380b917e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:51.231432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b98d08-a0d3-4bb1-a33d-bb1a5d7ba6a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:51.231958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b98d08-a0d3-4bb1-a33d-bb1a5d7ba6a5 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:51.232239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b98d08-a0d3-4bb1-a33d-bb1a5d7ba6a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:51.232540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b98d08-a0d3-4bb1-a33d-bb1a5d7ba6a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:51.234124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c1b98d08-a0d3-4bb1-a33d-bb1a5d7ba6a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:51.311895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:51.326793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b98d08-a0d3-4bb1-a33d-bb1a5d7ba6a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:51.328373 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b98d08-a0d3-4bb1-a33d-bb1a5d7ba6a5 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:51.347279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 720/1425] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:52.245821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03a93-debb-490c-b8d9-a0d54f66b3b9 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:52.246352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03a93-debb-490c-b8d9-a0d54f66b3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:52.246352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03a93-debb-490c-b8d9-a0d54f66b3b9 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:52.247692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:52.252679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca2b10-c093-435e-95c1-162a41d10260 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:52.253614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca2b10-c093-435e-95c1-162a41d10260 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:52.253896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca2b10-c093-435e-95c1-162a41d10260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:52.329973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0bca2b10-c093-435e-95c1-162a41d10260 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:52.331172 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0bca2b10-c093-435e-95c1-162a41d10260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:52.426630 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:52.574197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a03a93-debb-490c-b8d9-a0d54f66b3b9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:52.575764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-13a03a93-debb-490c-b8d9-a0d54f66b3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:52.658757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:52.722253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a03a93-debb-490c-b8d9-a0d54f66b3b9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:52.833551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984acd7-7ed5-4a3e-bbaf-651f20a1d7d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:52.834077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984acd7-7ed5-4a3e-bbaf-651f20a1d7d9 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:52.834359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984acd7-7ed5-4a3e-bbaf-651f20a1d7d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:52.834633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984acd7-7ed5-4a3e-bbaf-651f20a1d7d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:52.836290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1984acd7-7ed5-4a3e-bbaf-651f20a1d7d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:52.884200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:52.926941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984acd7-7ed5-4a3e-bbaf-651f20a1d7d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:52.928575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984acd7-7ed5-4a3e-bbaf-651f20a1d7d9 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:52.946286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 707/1428] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:53.762275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09602d3-f980-45a8-bf14-2b880df40b40 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:53.762806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09602d3-f980-45a8-bf14-2b880df40b40 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:53.762806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09602d3-f980-45a8-bf14-2b880df40b40 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:53.764135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:53.769704 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37931b-d5b1-4327-97d4-2ce8cfe28795 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:53.770136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37931b-d5b1-4327-97d4-2ce8cfe28795 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:53.770263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37931b-d5b1-4327-97d4-2ce8cfe28795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:53.845424 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-dc37931b-d5b1-4327-97d4-2ce8cfe28795 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:53.846878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dc37931b-d5b1-4327-97d4-2ce8cfe28795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:53.946364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:54.106289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09602d3-f980-45a8-bf14-2b880df40b40 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:54.107823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f09602d3-f980-45a8-bf14-2b880df40b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:54.156025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:54.257150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09602d3-f980-45a8-bf14-2b880df40b40 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:54.382420 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0b845-eff9-41ba-ac91-530110da7cfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:54.383024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0b845-eff9-41ba-ac91-530110da7cfd service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:54.383301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0b845-eff9-41ba-ac91-530110da7cfd service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:54.383577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0b845-eff9-41ba-ac91-530110da7cfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:54.385197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-36a0b845-eff9-41ba-ac91-530110da7cfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:54.392057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:54.475169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0b845-eff9-41ba-ac91-530110da7cfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:54.476787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0b845-eff9-41ba-ac91-530110da7cfd service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:54.494909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 723/1431] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:55.302290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9816ece-8f19-46ff-888c-8f75101a281c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:55.302818 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9816ece-8f19-46ff-888c-8f75101a281c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:55.302818 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9816ece-8f19-46ff-888c-8f75101a281c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:55.304113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:55.309038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831359f-3800-45aa-9873-c2c5bf99495a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:55.309452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831359f-3800-45aa-9873-c2c5bf99495a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:55.309592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831359f-3800-45aa-9873-c2c5bf99495a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:55.388669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c831359f-3800-45aa-9873-c2c5bf99495a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:55.389988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c831359f-3800-45aa-9873-c2c5bf99495a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:55.463711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:55.494025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:36:55.654128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9816ece-8f19-46ff-888c-8f75101a281c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:55.655657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a9816ece-8f19-46ff-888c-8f75101a281c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:55.674366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:55.808120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9816ece-8f19-46ff-888c-8f75101a281c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:55.894844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:55.920237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefc751-0be7-4d7f-957b-195aad5eedcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:55.920806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc751-0be7-4d7f-957b-195aad5eedcd service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:55.921085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc751-0be7-4d7f-957b-195aad5eedcd service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:55.921353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc751-0be7-4d7f-957b-195aad5eedcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:55.922943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-baefc751-0be7-4d7f-957b-195aad5eedcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:56.012733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefc751-0be7-4d7f-957b-195aad5eedcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:56.014337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefc751-0be7-4d7f-957b-195aad5eedcd service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:56.032297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 710/1434] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:56.835377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f88a34-b682-45d8-9dab-dea88119e506 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:56.835968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f88a34-b682-45d8-9dab-dea88119e506 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:56.835968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f88a34-b682-45d8-9dab-dea88119e506 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:56.837365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:56.842270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ea591-45a8-451b-b9bc-9ec8c2118c15 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:56.842745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ea591-45a8-451b-b9bc-9ec8c2118c15 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:56.842860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ea591-45a8-451b-b9bc-9ec8c2118c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:56.918240 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9b1ea591-45a8-451b-b9bc-9ec8c2118c15 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:56.919446 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9b1ea591-45a8-451b-b9bc-9ec8c2118c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:56.960849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:57.016391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:57.164623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f88a34-b682-45d8-9dab-dea88119e506 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:57.166188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-65f88a34-b682-45d8-9dab-dea88119e506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:57.316300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f88a34-b682-45d8-9dab-dea88119e506 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:57.371658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:57.431619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1784db5-9551-4b62-9d44-2ee544b153ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:57.432235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1784db5-9551-4b62-9d44-2ee544b153ce service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:57.432514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1784db5-9551-4b62-9d44-2ee544b153ce service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:57.432794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1784db5-9551-4b62-9d44-2ee544b153ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:57.434364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a1784db5-9551-4b62-9d44-2ee544b153ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:57.521418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1784db5-9551-4b62-9d44-2ee544b153ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:57.522988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1784db5-9551-4b62-9d44-2ee544b153ce service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:57.541251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 726/1437] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:36:58.344129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2c7cd-2c84-4d9d-ad87-0ceb2a271e48 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:58.344706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2c7cd-2c84-4d9d-ad87-0ceb2a271e48 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:58.344706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2c7cd-2c84-4d9d-ad87-0ceb2a271e48 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:58.346120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:36:58.351336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e51c-7a2c-4a62-ad2d-0259ea45baa9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:58.351760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e51c-7a2c-4a62-ad2d-0259ea45baa9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:58.351916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e51c-7a2c-4a62-ad2d-0259ea45baa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:58.428470 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9dd2e51c-7a2c-4a62-ad2d-0259ea45baa9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:58.429737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9dd2e51c-7a2c-4a62-ad2d-0259ea45baa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:58.431707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:58.528818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:58.677835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2c7cd-2c84-4d9d-ad87-0ceb2a271e48 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:58.679494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9ff2c7cd-2c84-4d9d-ad87-0ceb2a271e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:58.823870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2c7cd-2c84-4d9d-ad87-0ceb2a271e48 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:58.851236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:58.937022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600a910-bf26-4a78-8ac5-3965409c1f8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:58.937624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600a910-bf26-4a78-8ac5-3965409c1f8a service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:58.937904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600a910-bf26-4a78-8ac5-3965409c1f8a service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:58.938177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600a910-bf26-4a78-8ac5-3965409c1f8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:58.939859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8600a910-bf26-4a78-8ac5-3965409c1f8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:59.029398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600a910-bf26-4a78-8ac5-3965409c1f8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:59.031011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600a910-bf26-4a78-8ac5-3965409c1f8a service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:59.050229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 713/1440] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:00.886624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0ccf9-3d82-4493-b86e-eb95543b9c4d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:00.887184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0ccf9-3d82-4493-b86e-eb95543b9c4d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:00.887184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0ccf9-3d82-4493-b86e-eb95543b9c4d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:00.888519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:00.894524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dbef3-0e10-4933-9bc7-09ee1981cacf None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:00.895123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dbef3-0e10-4933-9bc7-09ee1981cacf None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:00.895123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dbef3-0e10-4933-9bc7-09ee1981cacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:00.941881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:00.967977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e14dbef3-0e10-4933-9bc7-09ee1981cacf None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:00.969243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e14dbef3-0e10-4933-9bc7-09ee1981cacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:01.066905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:01.349964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0ccf9-3d82-4493-b86e-eb95543b9c4d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:01.351481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-66a0ccf9-3d82-4493-b86e-eb95543b9c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:01.457941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0ccf9-3d82-4493-b86e-eb95543b9c4d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:01.490343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:01.531919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7501156-7973-4722-b332-aa44d8630cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:01.532537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7501156-7973-4722-b332-aa44d8630cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:01.532822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7501156-7973-4722-b332-aa44d8630cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:01.533093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7501156-7973-4722-b332-aa44d8630cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:01.534841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c7501156-7973-4722-b332-aa44d8630cf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:01.643823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7501156-7973-4722-b332-aa44d8630cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:01.645268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7501156-7973-4722-b332-aa44d8630cf7 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:01.663799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 729/1443] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:03.708218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e079-a2f6-441b-a99c-2a7827a3db86 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.708739 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e079-a2f6-441b-a99c-2a7827a3db86 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.708862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e079-a2f6-441b-a99c-2a7827a3db86 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.710180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:03.715393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70444a38-6ce4-4127-a7c6-9823a68aa78f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.715798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70444a38-6ce4-4127-a7c6-9823a68aa78f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.715937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70444a38-6ce4-4127-a7c6-9823a68aa78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.780858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-70444a38-6ce4-4127-a7c6-9823a68aa78f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:03.782041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-70444a38-6ce4-4127-a7c6-9823a68aa78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:03.795437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:03.880036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:04.162179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0e079-a2f6-441b-a99c-2a7827a3db86 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:04.164074 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bdf0e079-a2f6-441b-a99c-2a7827a3db86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:04.269305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0e079-a2f6-441b-a99c-2a7827a3db86 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:04.281179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:04.343122 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb509b-8938-47be-be7b-45b6191dfbc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:04.343728 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb509b-8938-47be-be7b-45b6191dfbc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:04.344020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb509b-8938-47be-be7b-45b6191dfbc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:04.344297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb509b-8938-47be-be7b-45b6191dfbc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:04.345981 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-deeb509b-8938-47be-be7b-45b6191dfbc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:04.453705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb509b-8938-47be-be7b-45b6191dfbc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:04.455754 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb509b-8938-47be-be7b-45b6191dfbc1 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:04.474221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 716/1446] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:06.700978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e74ebc-9084-4b16-91ee-1eb8815a19f9 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:06.701484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e74ebc-9084-4b16-91ee-1eb8815a19f9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:06.701595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e74ebc-9084-4b16-91ee-1eb8815a19f9 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:06.702837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:06.708931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466f23-0d75-489e-84aa-5dc0bb9a436c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:06.709399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466f23-0d75-489e-84aa-5dc0bb9a436c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:06.709506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466f23-0d75-489e-84aa-5dc0bb9a436c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:06.759495 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:06.777689 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5d466f23-0d75-489e-84aa-5dc0bb9a436c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:06.778908 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5d466f23-0d75-489e-84aa-5dc0bb9a436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:06.874217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:07.154960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e74ebc-9084-4b16-91ee-1eb8815a19f9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:07.156526 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-37e74ebc-9084-4b16-91ee-1eb8815a19f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:07.271477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e74ebc-9084-4b16-91ee-1eb8815a19f9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:07.312030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:07.347002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4a685-7e48-48c0-b51a-69be4a237144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:07.347612 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4a685-7e48-48c0-b51a-69be4a237144 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:07.347891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4a685-7e48-48c0-b51a-69be4a237144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:07.348161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4a685-7e48-48c0-b51a-69be4a237144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:07.349806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-97c4a685-7e48-48c0-b51a-69be4a237144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:07.461607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4a685-7e48-48c0-b51a-69be4a237144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:07.462961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4a685-7e48-48c0-b51a-69be4a237144 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:07.481355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 732/1449] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:09.542572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84c644-e5ff-4d68-bae0-3b4d2718cd4e None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:09.543133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84c644-e5ff-4d68-bae0-3b4d2718cd4e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:09.543133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84c644-e5ff-4d68-bae0-3b4d2718cd4e None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:09.544473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:09.549967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca5909-ce4b-4fad-82d5-c63b5a1c3346 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:09.550394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca5909-ce4b-4fad-82d5-c63b5a1c3346 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:09.550504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca5909-ce4b-4fad-82d5-c63b5a1c3346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:09.623666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-98ca5909-ce4b-4fad-82d5-c63b5a1c3346 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:09.624921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-98ca5909-ce4b-4fad-82d5-c63b5a1c3346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:09.646057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:09.725073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:10.009328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84c644-e5ff-4d68-bae0-3b4d2718cd4e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:10.010911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4d84c644-e5ff-4d68-bae0-3b4d2718cd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:10.115210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84c644-e5ff-4d68-bae0-3b4d2718cd4e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:10.135798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:10.191295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1b1fe-18cc-4e54-9130-f74bf165d8d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:10.191994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1b1fe-18cc-4e54-9130-f74bf165d8d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:10.192279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1b1fe-18cc-4e54-9130-f74bf165d8d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:10.192682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1b1fe-18cc-4e54-9130-f74bf165d8d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:10.194304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-79c1b1fe-18cc-4e54-9130-f74bf165d8d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:10.304027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1b1fe-18cc-4e54-9130-f74bf165d8d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:10.305424 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1b1fe-18cc-4e54-9130-f74bf165d8d2 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:10.324158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 719/1452] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:12.277869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a167ce-1d0e-4e9e-aba1-2e1e7e1e5ce9 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:12.278383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a167ce-1d0e-4e9e-aba1-2e1e7e1e5ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:12.278383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a167ce-1d0e-4e9e-aba1-2e1e7e1e5ce9 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:12.279689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:12.285734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16467739-b3cc-482f-84d7-38fed5445b5f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:12.286170 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16467739-b3cc-482f-84d7-38fed5445b5f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:12.286266 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16467739-b3cc-482f-84d7-38fed5445b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:12.350872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:12.353814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-16467739-b3cc-482f-84d7-38fed5445b5f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:12.355808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-16467739-b3cc-482f-84d7-38fed5445b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:12.457153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:12.737357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a167ce-1d0e-4e9e-aba1-2e1e7e1e5ce9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:12.738893 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b7a167ce-1d0e-4e9e-aba1-2e1e7e1e5ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:12.841033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a167ce-1d0e-4e9e-aba1-2e1e7e1e5ce9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:12.889824 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:12.914536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6bd45-b422-47ce-a601-7b7107c59d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:12.915207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6bd45-b422-47ce-a601-7b7107c59d28 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:12.915488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6bd45-b422-47ce-a601-7b7107c59d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:12.915753 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6bd45-b422-47ce-a601-7b7107c59d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:12.917360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-71c6bd45-b422-47ce-a601-7b7107c59d28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:13.025697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6bd45-b422-47ce-a601-7b7107c59d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:13.027046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6bd45-b422-47ce-a601-7b7107c59d28 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:13.044592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 735/1455] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:15.008794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737a58b-f65c-4eda-ac59-a79792800ca9 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:15.009355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737a58b-f65c-4eda-ac59-a79792800ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:15.009355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737a58b-f65c-4eda-ac59-a79792800ca9 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:15.010680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:15.015845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86acf9-3fcf-42ca-9473-dcd9cea39049 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:15.016268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86acf9-3fcf-42ca-9473-dcd9cea39049 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:15.016400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86acf9-3fcf-42ca-9473-dcd9cea39049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:15.081980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cc86acf9-3fcf-42ca-9473-dcd9cea39049 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:15.083213 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cc86acf9-3fcf-42ca-9473-dcd9cea39049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:15.112687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:15.179458 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:15.466901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737a58b-f65c-4eda-ac59-a79792800ca9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:15.468530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6737a58b-f65c-4eda-ac59-a79792800ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:15.571982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737a58b-f65c-4eda-ac59-a79792800ca9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:15.606620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:15.653979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ca2d6-8f2b-4878-9534-cda9c77e83bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:15.654587 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ca2d6-8f2b-4878-9534-cda9c77e83bc service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:15.654869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ca2d6-8f2b-4878-9534-cda9c77e83bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:15.655141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ca2d6-8f2b-4878-9534-cda9c77e83bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:15.656775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-711ca2d6-8f2b-4878-9534-cda9c77e83bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:15.765185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ca2d6-8f2b-4878-9534-cda9c77e83bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:15.766632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ca2d6-8f2b-4878-9534-cda9c77e83bc service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:15.784289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 722/1458] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:17.956162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a275e1d-883c-4958-a045-19f13d4c3ef3 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:17.956609 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a275e1d-883c-4958-a045-19f13d4c3ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:17.956841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a275e1d-883c-4958-a045-19f13d4c3ef3 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:17.958061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:17.963397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df819ba-9303-4645-8870-3bdcb9d923c9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:17.963797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df819ba-9303-4645-8870-3bdcb9d923c9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:17.963936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df819ba-9303-4645-8870-3bdcb9d923c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:18.028871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8df819ba-9303-4645-8870-3bdcb9d923c9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:18.030066 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8df819ba-9303-4645-8870-3bdcb9d923c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:18.038926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:18.126649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:37:18.407449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a275e1d-883c-4958-a045-19f13d4c3ef3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:18.409117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a275e1d-883c-4958-a045-19f13d4c3ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:18.513241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a275e1d-883c-4958-a045-19f13d4c3ef3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:18.568295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:18.587651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106679f-8718-4240-89d5-29dc0cc6be39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:18.588248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106679f-8718-4240-89d5-29dc0cc6be39 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:18.588553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106679f-8718-4240-89d5-29dc0cc6be39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:18.588831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106679f-8718-4240-89d5-29dc0cc6be39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:18.590409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d106679f-8718-4240-89d5-29dc0cc6be39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:18.699879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106679f-8718-4240-89d5-29dc0cc6be39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:18.701340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106679f-8718-4240-89d5-29dc0cc6be39 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:18.720013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 738/1461] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:20.668345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a55ed9-566f-43f9-ad27-befb74c326e9 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:20.668922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a55ed9-566f-43f9-ad27-befb74c326e9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:20.668922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a55ed9-566f-43f9-ad27-befb74c326e9 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:20.670279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:20.676868 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4277da3-9586-4940-b4af-ebd4c437b44e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:20.677322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4277da3-9586-4940-b4af-ebd4c437b44e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:20.677418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4277da3-9586-4940-b4af-ebd4c437b44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:20.744907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c4277da3-9586-4940-b4af-ebd4c437b44e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:20.746114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c4277da3-9586-4940-b4af-ebd4c437b44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:20.784394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:20.844339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:21.128757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a55ed9-566f-43f9-ad27-befb74c326e9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:21.130301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f6a55ed9-566f-43f9-ad27-befb74c326e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:21.233577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a55ed9-566f-43f9-ad27-befb74c326e9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:21.271785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:21.308050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a75e90-a132-4676-a506-8605c8dadfc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:21.308699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a75e90-a132-4676-a506-8605c8dadfc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:21.308986 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a75e90-a132-4676-a506-8605c8dadfc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:21.309265 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a75e90-a132-4676-a506-8605c8dadfc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:21.310920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-79a75e90-a132-4676-a506-8605c8dadfc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:21.420168 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a75e90-a132-4676-a506-8605c8dadfc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:21.421554 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a75e90-a132-4676-a506-8605c8dadfc5 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:21.439428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 725/1464] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:23.371129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef03cd-e659-4ed3-ba27-62befbabd716 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:23.371715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef03cd-e659-4ed3-ba27-62befbabd716 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:23.371715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef03cd-e659-4ed3-ba27-62befbabd716 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:23.373086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:23.379126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777993d-c4d3-4882-84ba-516c489286c1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:23.379563 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777993d-c4d3-4882-84ba-516c489286c1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:23.379686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777993d-c4d3-4882-84ba-516c489286c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:23.446355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a777993d-c4d3-4882-84ba-516c489286c1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:23.447565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a777993d-c4d3-4882-84ba-516c489286c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:23.465849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:23.545182 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:23.834770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef03cd-e659-4ed3-ba27-62befbabd716 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:23.836308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-51ef03cd-e659-4ed3-ba27-62befbabd716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:23.942774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef03cd-e659-4ed3-ba27-62befbabd716 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:24.009113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:24.016130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdf707-0767-437d-9b7a-bd68867bc1df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:24.017429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdf707-0767-437d-9b7a-bd68867bc1df service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:24.017653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdf707-0767-437d-9b7a-bd68867bc1df service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:24.017916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdf707-0767-437d-9b7a-bd68867bc1df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:24.019474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-94bdf707-0767-437d-9b7a-bd68867bc1df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:24.126265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdf707-0767-437d-9b7a-bd68867bc1df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:24.127575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdf707-0767-437d-9b7a-bd68867bc1df service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:24.146962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 741/1467] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:26.092917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04009b3a-5dd9-4813-b7b0-158aa73a83a0 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:26.093471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04009b3a-5dd9-4813-b7b0-158aa73a83a0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:26.093471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04009b3a-5dd9-4813-b7b0-158aa73a83a0 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:26.095386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:37:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:26.100746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5017873-f8a4-457d-b5be-b3450432ae57 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:26.101174 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5017873-f8a4-457d-b5be-b3450432ae57 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:26.101303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5017873-f8a4-457d-b5be-b3450432ae57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:26.169798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f5017873-f8a4-457d-b5be-b3450432ae57 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:26.170995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f5017873-f8a4-457d-b5be-b3450432ae57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:26.222276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:26.272370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:26.557307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04009b3a-5dd9-4813-b7b0-158aa73a83a0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:26.558849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-04009b3a-5dd9-4813-b7b0-158aa73a83a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:26.661814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04009b3a-5dd9-4813-b7b0-158aa73a83a0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:26.709748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:26.736337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7ff2d-5b17-4679-a11f-e99371809b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:26.736964 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7ff2d-5b17-4679-a11f-e99371809b40 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:26.737239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7ff2d-5b17-4679-a11f-e99371809b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:26.737511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7ff2d-5b17-4679-a11f-e99371809b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:26.739132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-53e7ff2d-5b17-4679-a11f-e99371809b40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:26.968947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7ff2d-5b17-4679-a11f-e99371809b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:26.970975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7ff2d-5b17-4679-a11f-e99371809b40 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:26.988433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 728/1470] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:36.270680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=22 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:36.986291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=66 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:46.666661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071f4f-942f-4c52-832f-2a52ba66663d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:46.667238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071f4f-942f-4c52-832f-2a52ba66663d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:46.667238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071f4f-942f-4c52-832f-2a52ba66663d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:46.668593 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 743/1471] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:46.674361 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fda32-bf7a-474d-ab1b-07d71d4de9ee None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:46.674785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fda32-bf7a-474d-ab1b-07d71d4de9ee None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:46.674925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fda32-bf7a-474d-ab1b-07d71d4de9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:46.741909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2a6fda32-bf7a-474d-ab1b-07d71d4de9ee None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:46.743098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2a6fda32-bf7a-474d-ab1b-07d71d4de9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:46.837530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 729/1472] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:47.138584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707be1c-eb4c-4189-aede-1103bc1ed713 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:47.138869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707be1c-eb4c-4189-aede-1103bc1ed713 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:47.139141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707be1c-eb4c-4189-aede-1103bc1ed713 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:47.140571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 744/1473] 10.222.0.60 () {54 vars in 967 bytes} [Wed Aug 31 11:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:47.145231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3eec5-ec36-497e-a279-afe35ae0556b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:47.145520 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3eec5-ec36-497e-a279-afe35ae0556b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:47.145795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3eec5-ec36-497e-a279-afe35ae0556b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:47.221178 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-86d3eec5-ec36-497e-a279-afe35ae0556b None None] MFA Rules not processed for user `18ec048788904bc58245e878a8644091`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:47.222426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-86d3eec5-ec36-497e-a279-afe35ae0556b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:47.243162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:47.298281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 730/1474] 10.222.0.60 () {58 vars in 1064 bytes} [Wed Aug 31 11:37:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:47.305682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707be1c-eb4c-4189-aede-1103bc1ed713 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:47.307215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4707be1c-eb4c-4189-aede-1103bc1ed713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:47.426929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707be1c-eb4c-4189-aede-1103bc1ed713 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:47.499516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1859-ef55-464e-af8b-691e7fddba55 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ec048788904bc58245e878a8644091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:47.500113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1859-ef55-464e-af8b-691e7fddba55 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:47.500393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1859-ef55-464e-af8b-691e7fddba55 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:47.500761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1859-ef55-464e-af8b-691e7fddba55 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:47.502344 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e83c1859-ef55-464e-af8b-691e7fddba55 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:47.565743 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:47.620362 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c1859-ef55-464e-af8b-691e7fddba55 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:47.621808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c1859-ef55-464e-af8b-691e7fddba55 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:47.640859 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 745/1475] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:49.616465 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb0b5f-c9ca-453d-8a81-154b9b9dca90 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:49.617083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb0b5f-c9ca-453d-8a81-154b9b9dca90 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:49.617083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb0b5f-c9ca-453d-8a81-154b9b9dca90 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:49.618387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 731/1476] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:37:49.623504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082f637-6ed9-4c87-a245-03fba076c2de None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:49.623931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082f637-6ed9-4c87-a245-03fba076c2de None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:49.624066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082f637-6ed9-4c87-a245-03fba076c2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:49.693228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5082f637-6ed9-4c87-a245-03fba076c2de None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:49.694705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5082f637-6ed9-4c87-a245-03fba076c2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:49.792956 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 746/1477] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:50.064523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb0b5f-c9ca-453d-8a81-154b9b9dca90 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:50.066062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-55fb0b5f-c9ca-453d-8a81-154b9b9dca90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:50.188666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb0b5f-c9ca-453d-8a81-154b9b9dca90 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:50.238169 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:50.283465 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1513e-139b-4e5d-aa2d-db02eb19eb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:50.284065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1513e-139b-4e5d-aa2d-db02eb19eb10 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:50.284339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1513e-139b-4e5d-aa2d-db02eb19eb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:50.284647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1513e-139b-4e5d-aa2d-db02eb19eb10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:50.285278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1513e-139b-4e5d-aa2d-db02eb19eb10 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:50.286995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-08c1513e-139b-4e5d-aa2d-db02eb19eb10 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:37:50.286995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 31 11:37:50.286995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: Aug 31 11:37:50.288112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1513e-139b-4e5d-aa2d-db02eb19eb10 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:50.306813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 732/1478] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Aug 31 11:37:50 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:37:52.257655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2a456-1712-405b-8568-f3f64c378492 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:52.258231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2a456-1712-405b-8568-f3f64c378492 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:52.258231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2a456-1712-405b-8568-f3f64c378492 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:52.259452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 747/1479] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:52.264842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43d030-3911-4e07-979f-f7ecf1828e4f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:52.265373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43d030-3911-4e07-979f-f7ecf1828e4f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:52.265694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43d030-3911-4e07-979f-f7ecf1828e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:52.332247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-be43d030-3911-4e07-979f-f7ecf1828e4f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:52.333539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-be43d030-3911-4e07-979f-f7ecf1828e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:52.431907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 733/1480] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:52.730698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2a456-1712-405b-8568-f3f64c378492 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:52.732246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-78e2a456-1712-405b-8568-f3f64c378492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:52.742843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:52.886090 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2a456-1712-405b-8568-f3f64c378492 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:52.966653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:52.999770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7283347-f193-401f-bc0b-d6389b4bbfbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:53.000424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7283347-f193-401f-bc0b-d6389b4bbfbd service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:53.000738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7283347-f193-401f-bc0b-d6389b4bbfbd service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:53.001013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7283347-f193-401f-bc0b-d6389b4bbfbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:53.002642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f7283347-f193-401f-bc0b-d6389b4bbfbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:53.093775 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7283347-f193-401f-bc0b-d6389b4bbfbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:53.095369 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7283347-f193-401f-bc0b-d6389b4bbfbd service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:53.113253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 748/1481] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:55.200857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef5715-9f0b-4bba-9505-9232dfc10a3a None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:55.201410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef5715-9f0b-4bba-9505-9232dfc10a3a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:55.201410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef5715-9f0b-4bba-9505-9232dfc10a3a None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:55.202785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 734/1482] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:55.208060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37668c5d-8fb7-4e9d-8667-b588fb22e78a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:55.208508 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37668c5d-8fb7-4e9d-8667-b588fb22e78a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:55.208684 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37668c5d-8fb7-4e9d-8667-b588fb22e78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:55.275489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-37668c5d-8fb7-4e9d-8667-b588fb22e78a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:55.276711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-37668c5d-8fb7-4e9d-8667-b588fb22e78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:55.322800 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:55.373481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 749/1483] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:37:55.662930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef5715-9f0b-4bba-9505-9232dfc10a3a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:55.664468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-87ef5715-9f0b-4bba-9505-9232dfc10a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:55.688892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:55.781268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef5715-9f0b-4bba-9505-9232dfc10a3a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:55.864924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e2b1-bdce-4d2b-92fe-f944bcb58098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:55.865523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e2b1-bdce-4d2b-92fe-f944bcb58098 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:55.865803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e2b1-bdce-4d2b-92fe-f944bcb58098 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:55.866077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e2b1-bdce-4d2b-92fe-f944bcb58098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:55.867667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ea37e2b1-bdce-4d2b-92fe-f944bcb58098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:55.943295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:55.980277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37e2b1-bdce-4d2b-92fe-f944bcb58098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:55.981761 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37e2b1-bdce-4d2b-92fe-f944bcb58098 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:56.000435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 735/1484] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:57.958753 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fb713-40cd-49fb-8053-24407e1a894c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:57.959306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fb713-40cd-49fb-8053-24407e1a894c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:57.959306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fb713-40cd-49fb-8053-24407e1a894c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:57.960572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 750/1485] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:37:57.965740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df4307-8cbc-4118-9c3b-6e7062d9e402 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:57.966180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df4307-8cbc-4118-9c3b-6e7062d9e402 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:57.966295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df4307-8cbc-4118-9c3b-6e7062d9e402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:58.033322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f4df4307-8cbc-4118-9c3b-6e7062d9e402 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:58.034529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f4df4307-8cbc-4118-9c3b-6e7062d9e402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:58.132959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 736/1486] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:58.425480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fb713-40cd-49fb-8053-24407e1a894c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:58.427010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-321fb713-40cd-49fb-8053-24407e1a894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:58.530596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fb713-40cd-49fb-8053-24407e1a894c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:58.600368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:58.602309 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcef70-8d27-458e-be3c-11040f0eaa90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:58.603110 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcef70-8d27-458e-be3c-11040f0eaa90 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:58.603568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcef70-8d27-458e-be3c-11040f0eaa90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:58.603882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcef70-8d27-458e-be3c-11040f0eaa90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:58.605863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2cbcef70-8d27-458e-be3c-11040f0eaa90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:58.715039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcef70-8d27-458e-be3c-11040f0eaa90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:58.716482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcef70-8d27-458e-be3c-11040f0eaa90 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:58.735419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 751/1487] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:38:00.846075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f9c49-f7c0-456d-8e71-0153a707a105 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:38:00.846664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f9c49-f7c0-456d-8e71-0153a707a105 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:38:00.846664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f9c49-f7c0-456d-8e71-0153a707a105 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:38:00.848041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 737/1488] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:38:00.853186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dd195-8369-4b46-8852-7f34bb7e82b9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:38:00.853597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dd195-8369-4b46-8852-7f34bb7e82b9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:38:00.853727 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dd195-8369-4b46-8852-7f34bb7e82b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:38:00.920033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ed5dd195-8369-4b46-8852-7f34bb7e82b9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:38:00.921774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ed5dd195-8369-4b46-8852-7f34bb7e82b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:38:00.974635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:38:01.018603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 752/1489] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:38:01.312451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f9c49-f7c0-456d-8e71-0153a707a105 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:38:01.314012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-702f9c49-f7c0-456d-8e71-0153a707a105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:38:01.350764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:38:01.418614 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f9c49-f7c0-456d-8e71-0153a707a105 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:38:01.493609 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ba8be-9c8c-4e02-8607-062f698a4f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:38:01.494222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ba8be-9c8c-4e02-8607-062f698a4f35 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:38:01.494506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ba8be-9c8c-4e02-8607-062f698a4f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:38:01.494780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ba8be-9c8c-4e02-8607-062f698a4f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:38:01.496383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-608ba8be-9c8c-4e02-8607-062f698a4f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:38:01.584339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:38:01.613884 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ba8be-9c8c-4e02-8607-062f698a4f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:38:01.615338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ba8be-9c8c-4e02-8607-062f698a4f35 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:38:01.637133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 738/1490] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:38:03.761173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a90e6-0e6c-4e70-af66-1ca66a8e8616 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:38:03.761692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a90e6-0e6c-4e70-af66-1ca66a8e8616 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:38:03.761805 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a90e6-0e6c-4e70-af66-1ca66a8e8616 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:38:03.763059 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 753/1491] 10.222.0.60 () {54 vars in 927 bytes} [Wed Aug 31 11:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:38:03.768321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb6d9b-ef97-49f3-a105-a937adabf1c9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:38:03.768775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb6d9b-ef97-49f3-a105-a937adabf1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:38:03.768914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb6d9b-ef97-49f3-a105-a937adabf1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:38:03.834241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-dceb6d9b-ef97-49f3-a105-a937adabf1c9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:38:03.835452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dceb6d9b-ef97-49f3-a105-a937adabf1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:38:03.937051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 739/1492] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Aug 31 11:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:38:04.224026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a90e6-0e6c-4e70-af66-1ca66a8e8616 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:38:04.225628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c75a90e6-0e6c-4e70-af66-1ca66a8e8616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:38:04.328692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a90e6-0e6c-4e70-af66-1ca66a8e8616 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:38:04.402698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ded43-3f89-4d1a-8be3-041547008efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:38:04.403301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ded43-3f89-4d1a-8be3-041547008efb service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:38:04.403584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ded43-3f89-4d1a-8be3-041547008efb service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:38:04.403856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ded43-3f89-4d1a-8be3-041547008efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:38:04.406165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0c3ded43-3f89-4d1a-8be3-041547008efb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:38:04.415187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:38:04.517001 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ded43-3f89-4d1a-8be3-041547008efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:38:04.518353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ded43-3f89-4d1a-8be3-041547008efb service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:38:04.536026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 754/1493] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:38:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:38:13.934575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=82 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:38:14.533798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=50 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:17.938408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a336b-5ce6-4022-875d-4666e90eb97c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:17.939014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a336b-5ce6-4022-875d-4666e90eb97c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:17.939014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a336b-5ce6-4022-875d-4666e90eb97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:18.015559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e09a336b-5ce6-4022-875d-4666e90eb97c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:18.016793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e09a336b-5ce6-4022-875d-4666e90eb97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:18.113941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 740/1494] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:18.122346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738b43e-972f-45d8-b3c2-26fc74928607 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:18.122838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738b43e-972f-45d8-b3c2-26fc74928607 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:18.122951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738b43e-972f-45d8-b3c2-26fc74928607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:18.201765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5738b43e-972f-45d8-b3c2-26fc74928607 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:18.202882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5738b43e-972f-45d8-b3c2-26fc74928607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:18.298321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 755/1495] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:18.302902 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a336b-5ce6-4022-875d-4666e90eb97c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:18.304108 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e09a336b-5ce6-4022-875d-4666e90eb97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:18.316028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a336b-5ce6-4022-875d-4666e90eb97c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:18.376731 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:18.410417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7e50d-4d19-4f6d-8af8-be43528ea299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:18.411015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7e50d-4d19-4f6d-8af8-be43528ea299 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:18.411290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7e50d-4d19-4f6d-8af8-be43528ea299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:18.411555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7e50d-4d19-4f6d-8af8-be43528ea299 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:18.412188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7e50d-4d19-4f6d-8af8-be43528ea299 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:39:18.412505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7e50d-4d19-4f6d-8af8-be43528ea299 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:18.413516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-56d7e50d-4d19-4f6d-8af8-be43528ea299 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:39:18.413516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 31 11:39:18.413516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: Aug 31 11:39:18.414390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7e50d-4d19-4f6d-8af8-be43528ea299 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:18.425260 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 741/1496] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:39:18.434180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738b43e-972f-45d8-b3c2-26fc74928607 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:18.443579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738b43e-972f-45d8-b3c2-26fc74928607 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:18.504039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:18.541256 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115df71e-9b79-4aa0-bd86-88174ae9a1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:18.543393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115df71e-9b79-4aa0-bd86-88174ae9a1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:18.543681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115df71e-9b79-4aa0-bd86-88174ae9a1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:18.543963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115df71e-9b79-4aa0-bd86-88174ae9a1a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:18.544650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115df71e-9b79-4aa0-bd86-88174ae9a1a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:18.546419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-115df71e-9b79-4aa0-bd86-88174ae9a1a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:39:18.546419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 31 11:39:18.546419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: Aug 31 11:39:18.547632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115df71e-9b79-4aa0-bd86-88174ae9a1a0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:18.747370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 756/1497] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:18 2022] POST /identity/v3/projects => generated 379 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:18.751555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7e50d-4d19-4f6d-8af8-be43528ea299 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:18.753091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-56d7e50d-4d19-4f6d-8af8-be43528ea299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:18.787002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7e50d-4d19-4f6d-8af8-be43528ea299 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:18.886821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce3c5d-8767-4cad-99c0-433fe12e0e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:18.887746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce3c5d-8767-4cad-99c0-433fe12e0e01 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:18.888031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce3c5d-8767-4cad-99c0-433fe12e0e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:18.888303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce3c5d-8767-4cad-99c0-433fe12e0e01 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:18.888986 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce3c5d-8767-4cad-99c0-433fe12e0e01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:18.890003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-5dce3c5d-8767-4cad-99c0-433fe12e0e01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:39:18.890003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 31 11:39:18.890003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: Aug 31 11:39:18.891213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce3c5d-8767-4cad-99c0-433fe12e0e01 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:18.935982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 742/1498] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:18 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:18.940095 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115df71e-9b79-4aa0-bd86-88174ae9a1a0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:18.941699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-115df71e-9b79-4aa0-bd86-88174ae9a1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:19.004910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115df71e-9b79-4aa0-bd86-88174ae9a1a0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:19.092611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:19.106295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646a4df-a744-4c2e-84f5-4c245f824535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:19.107844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646a4df-a744-4c2e-84f5-4c245f824535 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:19.108156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646a4df-a744-4c2e-84f5-4c245f824535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:19.108431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646a4df-a744-4c2e-84f5-4c245f824535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:19.109116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a646a4df-a744-4c2e-84f5-4c245f824535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:19.110128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-a646a4df-a744-4c2e-84f5-4c245f824535 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:39:19.110128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 31 11:39:19.110128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: Aug 31 11:39:19.111168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a646a4df-a744-4c2e-84f5-4c245f824535 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:19.137002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 757/1499] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:19.141153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce3c5d-8767-4cad-99c0-433fe12e0e01 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:19.142694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5dce3c5d-8767-4cad-99c0-433fe12e0e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:19.156346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce3c5d-8767-4cad-99c0-433fe12e0e01 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:19.185287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:19.274767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3ae9f-8164-4819-9766-dec0b1c0fa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:19.275473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3ae9f-8164-4819-9766-dec0b1c0fa25 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:19.275757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3ae9f-8164-4819-9766-dec0b1c0fa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:19.276050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3ae9f-8164-4819-9766-dec0b1c0fa25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:19.276892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3ae9f-8164-4819-9766-dec0b1c0fa25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:19.277994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-f6c3ae9f-8164-4819-9766-dec0b1c0fa25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:39:19.277994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 31 11:39:19.277994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: Aug 31 11:39:19.278941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3ae9f-8164-4819-9766-dec0b1c0fa25 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:19.292623 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 743/1500] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:19.297638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646a4df-a744-4c2e-84f5-4c245f824535 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:19.299540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a646a4df-a744-4c2e-84f5-4c245f824535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:19.338927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646a4df-a744-4c2e-84f5-4c245f824535 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:19.445597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63e3f8-50f3-4f35-8bcf-ec8ce8cd745e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:19.446254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63e3f8-50f3-4f35-8bcf-ec8ce8cd745e admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:19.446531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63e3f8-50f3-4f35-8bcf-ec8ce8cd745e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:19.446804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63e3f8-50f3-4f35-8bcf-ec8ce8cd745e admin admin] PATH_INFO: `/v3/projects/4f097bd7dfa04f6b8649f44d58b93bfe/users/f22bd1b2c5734c1db074f3fedaa3424c/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:19.476439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63e3f8-50f3-4f35-8bcf-ec8ce8cd745e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f097bd7dfa04f6b8649f44d58b93bfe, user_id=f22bd1b2c5734c1db074f3fedaa3424c, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:19.477502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-be63e3f8-50f3-4f35-8bcf-ec8ce8cd745e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:39:19.477502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 31 11:39:19.477502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: Aug 31 11:39:19.480269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63e3f8-50f3-4f35-8bcf-ec8ce8cd745e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:19.509810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 758/1501] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:19 2022] PUT /identity/v3/projects/4f097bd7dfa04f6b8649f44d58b93bfe/users/f22bd1b2c5734c1db074f3fedaa3424c/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:19.522276 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597b43e-b7f4-437c-b85a-dbd487800a9b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:19.522590 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597b43e-b7f4-437c-b85a-dbd487800a9b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:19.522869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597b43e-b7f4-437c-b85a-dbd487800a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:19.591814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e597b43e-b7f4-437c-b85a-dbd487800a9b None None] MFA Rules not processed for user `f22bd1b2c5734c1db074f3fedaa3424c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:19.593069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e597b43e-b7f4-437c-b85a-dbd487800a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:19.688576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 744/1502] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:39:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:19.702914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83aecdd-aac7-471a-8798-1c05b8ac1876 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:19.703203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83aecdd-aac7-471a-8798-1c05b8ac1876 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:19.703479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83aecdd-aac7-471a-8798-1c05b8ac1876 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:19.704861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 759/1503] 10.222.0.60 () {54 vars in 964 bytes} [Wed Aug 31 11:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:39:19.709800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b91db4-4e19-460b-8cb8-8c3f9dd9e3e8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:19.710081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b91db4-4e19-460b-8cb8-8c3f9dd9e3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:19.710381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b91db4-4e19-460b-8cb8-8c3f9dd9e3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:19.743243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:19.798535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-70b91db4-4e19-460b-8cb8-8c3f9dd9e3e8 None None] MFA Rules not processed for user `e8dec99551ff4c31a17247cde3515acd`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:19.799791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-70b91db4-4e19-460b-8cb8-8c3f9dd9e3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:19.877197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 745/1504] 10.222.0.60 () {58 vars in 1061 bytes} [Wed Aug 31 11:39:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:19.887243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83aecdd-aac7-471a-8798-1c05b8ac1876 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:19.888812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c83aecdd-aac7-471a-8798-1c05b8ac1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:19.940521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:19.996217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83aecdd-aac7-471a-8798-1c05b8ac1876 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:20.081432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998bb52-1bcf-406b-a616-48ff5ae2fe9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:20.082041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998bb52-1bcf-406b-a616-48ff5ae2fe9b service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:20.082322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998bb52-1bcf-406b-a616-48ff5ae2fe9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:20.082606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998bb52-1bcf-406b-a616-48ff5ae2fe9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:20.084313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8998bb52-1bcf-406b-a616-48ff5ae2fe9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:20.183164 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998bb52-1bcf-406b-a616-48ff5ae2fe9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:20.184621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998bb52-1bcf-406b-a616-48ff5ae2fe9b service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:20.188149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:20.204175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 760/1505] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:20.255453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07be69-f208-4e25-95ec-92bd3243f52d None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:20.255892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07be69-f208-4e25-95ec-92bd3243f52d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:20.256006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07be69-f208-4e25-95ec-92bd3243f52d None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:20.257290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 746/1506] 10.222.0.60 () {52 vars in 830 bytes} [Wed Aug 31 11:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:39:20.261046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998bb52-1bcf-406b-a616-48ff5ae2fe9b service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:20.262235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8998bb52-1bcf-406b-a616-48ff5ae2fe9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:20.362552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998bb52-1bcf-406b-a616-48ff5ae2fe9b service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:20.456542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:20.457095 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:20.457374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:20.457644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] PATH_INFO: `/v3/users/f22bd1b2c5734c1db074f3fedaa3424c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:20.470035 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f22bd1b2c5734c1db074f3fedaa3424c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:20.471043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:39:20.471043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 31 11:39:20.471043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: Aug 31 11:39:20.472261 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:20.482656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:20.538395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f22bd1b2c5734c1db074f3fedaa3424c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:20.552662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f22bd1b2c5734c1db074f3fedaa3424c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:20.553213 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f22bd1b2c5734c1db074f3fedaa3424c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:20.553640 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f22bd1b2c5734c1db074f3fedaa3424c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:20.563844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f22bd1b2c5734c1db074f3fedaa3424c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:20.592926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 761/1507] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:20 2022] DELETE /identity/v3/users/f22bd1b2c5734c1db074f3fedaa3424c => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:20.597274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60a05f-9b27-4bcb-8fa2-db3b4a29327a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:20.597589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60a05f-9b27-4bcb-8fa2-db3b4a29327a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:20.597871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60a05f-9b27-4bcb-8fa2-db3b4a29327a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:20.654476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8c60a05f-9b27-4bcb-8fa2-db3b4a29327a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:20.655679 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8c60a05f-9b27-4bcb-8fa2-db3b4a29327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:20.734695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:20.747344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 747/1508] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:20.757079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:20.758321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:20.858188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e4e0f-76ae-4c84-8606-994cfeb785d0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:20.913131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:20.934129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f392c5-d81a-44b2-b4f1-fde0c4949d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:20.934747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f392c5-d81a-44b2-b4f1-fde0c4949d0a service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:20.935030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f392c5-d81a-44b2-b4f1-fde0c4949d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:20.935303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f392c5-d81a-44b2-b4f1-fde0c4949d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:20.936967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-11f392c5-d81a-44b2-b4f1-fde0c4949d0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:21.047545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f392c5-d81a-44b2-b4f1-fde0c4949d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:21.048933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f392c5-d81a-44b2-b4f1-fde0c4949d0a service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:21.068193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 762/1509] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:21.428522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60a05f-9b27-4bcb-8fa2-db3b4a29327a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:21.428522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8c60a05f-9b27-4bcb-8fa2-db3b4a29327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:21.440365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60a05f-9b27-4bcb-8fa2-db3b4a29327a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:21.536159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:21.536830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:21.537115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:21.537388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] PATH_INFO: `/v3/projects/4f097bd7dfa04f6b8649f44d58b93bfe` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:21.547297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f097bd7dfa04f6b8649f44d58b93bfe)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:21.548327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:39:21.548327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 31 11:39:21.548327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: Aug 31 11:39:21.549563 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:21.607657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f097bd7dfa04f6b8649f44d58b93bfe', 'request_id': 'req-e1c79c43-440f-49e4-89fe-9f730f2be577'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:21.613695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f097bd7dfa04f6b8649f44d58b93bfe', 'request_id': 'req-e1c79c43-440f-49e4-89fe-9f730f2be577'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:21.627533 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f097bd7dfa04f6b8649f44d58b93bfe', 'request_id': 'req-e1c79c43-440f-49e4-89fe-9f730f2be577'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:21.653437 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] The token cache is being invalidate because project 4f097bd7dfa04f6b8649f44d58b93bfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:21.653734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:21.654258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e1c79c43-440f-49e4-89fe-9f730f2be577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:21.655562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 748/1510] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:21 2022] DELETE /identity/v3/projects/4f097bd7dfa04f6b8649f44d58b93bfe => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:31.065710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=62 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:31.651502 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:54.671471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98db02e-b7ca-4610-8168-2f073b35231c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:54.672094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98db02e-b7ca-4610-8168-2f073b35231c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:54.672094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98db02e-b7ca-4610-8168-2f073b35231c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:54.673592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:40:54.678628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd44b0a-c40b-40f9-8feb-d15f33bc31df None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:54.679074 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd44b0a-c40b-40f9-8feb-d15f33bc31df None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:54.679226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd44b0a-c40b-40f9-8feb-d15f33bc31df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:54.760213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-fdd44b0a-c40b-40f9-8feb-d15f33bc31df None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:54.761473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fdd44b0a-c40b-40f9-8feb-d15f33bc31df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:54.866544 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:55.028266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98db02e-b7ca-4610-8168-2f073b35231c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:55.030361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e98db02e-b7ca-4610-8168-2f073b35231c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:55.149338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98db02e-b7ca-4610-8168-2f073b35231c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:55.154669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03e987-4262-4fb2-b20a-7809f9482d07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:55.155247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03e987-4262-4fb2-b20a-7809f9482d07 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:55.155525 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03e987-4262-4fb2-b20a-7809f9482d07 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:55.155796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03e987-4262-4fb2-b20a-7809f9482d07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:55.163567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03e987-4262-4fb2-b20a-7809f9482d07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:55.165155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03e987-4262-4fb2-b20a-7809f9482d07 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:55.175064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 764/1513] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:56.056512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe73cd-2d62-4359-a52a-a720e5d53dc5 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:56.057029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe73cd-2d62-4359-a52a-a720e5d53dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:56.057029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe73cd-2d62-4359-a52a-a720e5d53dc5 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:56.058320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:40:56.063075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0df37-c5df-4faa-9583-e5f72c877a85 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:56.063549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0df37-c5df-4faa-9583-e5f72c877a85 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:56.063656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0df37-c5df-4faa-9583-e5f72c877a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:56.083935 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-edf0df37-c5df-4faa-9583-e5f72c877a85 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:56.085166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-edf0df37-c5df-4faa-9583-e5f72c877a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:56.092921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:56.247981 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe73cd-2d62-4359-a52a-a720e5d53dc5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:56.262206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe73cd-2d62-4359-a52a-a720e5d53dc5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:56.268255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b4b0e-8a9c-4593-acb6-bf443ca2692c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:56.270235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b4b0e-8a9c-4593-acb6-bf443ca2692c service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:56.270654 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b4b0e-8a9c-4593-acb6-bf443ca2692c service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:56.271589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b4b0e-8a9c-4593-acb6-bf443ca2692c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:56.281289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b4b0e-8a9c-4593-acb6-bf443ca2692c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:56.283561 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b4b0e-8a9c-4593-acb6-bf443ca2692c service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:56.293484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 751/1516] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:56.828903 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221034d5-e49c-4b48-8fb4-f06e26ce2212 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:56.829194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221034d5-e49c-4b48-8fb4-f06e26ce2212 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:56.829456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221034d5-e49c-4b48-8fb4-f06e26ce2212 None None] PATH_INFO: `/v3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:56.830762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 766/1517] 10.222.0.39 () {54 vars in 925 bytes} [Wed Aug 31 11:40:56 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:40:56.837382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c79c42-2dbd-4bf5-9f57-ba0719d68132 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:56.837671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c79c42-2dbd-4bf5-9f57-ba0719d68132 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:56.837969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c79c42-2dbd-4bf5-9f57-ba0719d68132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:56.896434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-59c79c42-2dbd-4bf5-9f57-ba0719d68132 None None] MFA Rules not processed for user `f9114a71a3a247188b263d87b6cb12c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:56.897682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-59c79c42-2dbd-4bf5-9f57-ba0719d68132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:56.914093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:56.971496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 752/1518] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Aug 31 11:40:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:00.894399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f282e0-800b-435d-8679-7ff4e6c6e257 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:00.894929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f282e0-800b-435d-8679-7ff4e6c6e257 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:00.894929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f282e0-800b-435d-8679-7ff4e6c6e257 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:00.896171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 767/1519] 10.222.0.60 () {54 vars in 961 bytes} [Wed Aug 31 11:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:41:00.901612 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716782a-b434-4c27-a1d3-b375949b75c5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:00.902041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716782a-b434-4c27-a1d3-b375949b75c5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:00.902183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716782a-b434-4c27-a1d3-b375949b75c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:00.922480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5716782a-b434-4c27-a1d3-b375949b75c5 None None] MFA Rules not processed for user `f9114a71a3a247188b263d87b6cb12c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:00.924287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5716782a-b434-4c27-a1d3-b375949b75c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:00.933063 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 753/1520] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Aug 31 11:41:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:00.941519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f282e0-800b-435d-8679-7ff4e6c6e257 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:00.956524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f282e0-800b-435d-8679-7ff4e6c6e257 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:00.961847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fda43d-4441-4eae-935f-029387fa6c59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:00.962457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fda43d-4441-4eae-935f-029387fa6c59 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:00.962783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fda43d-4441-4eae-935f-029387fa6c59 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:00.963056 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fda43d-4441-4eae-935f-029387fa6c59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:00.973199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fda43d-4441-4eae-935f-029387fa6c59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:00.974795 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fda43d-4441-4eae-935f-029387fa6c59 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:00.984410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 768/1521] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:07.506853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc469ab-e79d-492d-8d75-dff2d3091562 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:07.507433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc469ab-e79d-492d-8d75-dff2d3091562 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:07.507433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc469ab-e79d-492d-8d75-dff2d3091562 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:07.508769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:41:07.513584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ffeb0-7e59-4a4d-b272-23b5aa2c4849 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:07.514003 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ffeb0-7e59-4a4d-b272-23b5aa2c4849 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:07.514139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ffeb0-7e59-4a4d-b272-23b5aa2c4849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:07.533983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d27ffeb0-7e59-4a4d-b272-23b5aa2c4849 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:07.535192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d27ffeb0-7e59-4a4d-b272-23b5aa2c4849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:07.543029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:41:07.703626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc469ab-e79d-492d-8d75-dff2d3091562 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:07.718044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc469ab-e79d-492d-8d75-dff2d3091562 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:07.724048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46e08a-9aee-4667-87c7-22ddec4dc984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:07.724686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46e08a-9aee-4667-87c7-22ddec4dc984 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:07.724970 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46e08a-9aee-4667-87c7-22ddec4dc984 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:07.725249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46e08a-9aee-4667-87c7-22ddec4dc984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:07.733177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46e08a-9aee-4667-87c7-22ddec4dc984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:07.734769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46e08a-9aee-4667-87c7-22ddec4dc984 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:07.744701 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 755/1524] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:08.591715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144674-8384-4da1-94ff-bd89657bb3e6 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:08.592241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144674-8384-4da1-94ff-bd89657bb3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:08.592357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144674-8384-4da1-94ff-bd89657bb3e6 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:08.593650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:41:08.598876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d8a7-b7e4-40d6-a8cf-ce02c145285b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:08.599290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d8a7-b7e4-40d6-a8cf-ce02c145285b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:08.599428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d8a7-b7e4-40d6-a8cf-ce02c145285b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:08.619813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1ac7d8a7-b7e4-40d6-a8cf-ce02c145285b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:08.621105 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1ac7d8a7-b7e4-40d6-a8cf-ce02c145285b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:08.629551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:08.782408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3144674-8384-4da1-94ff-bd89657bb3e6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:08.796783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3144674-8384-4da1-94ff-bd89657bb3e6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:08.802568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58dcc6-c36c-417f-81f7-545a8eeb17b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:08.803153 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58dcc6-c36c-417f-81f7-545a8eeb17b1 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:08.803428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58dcc6-c36c-417f-81f7-545a8eeb17b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:08.803696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58dcc6-c36c-417f-81f7-545a8eeb17b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:08.811487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58dcc6-c36c-417f-81f7-545a8eeb17b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:08.813070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58dcc6-c36c-417f-81f7-545a8eeb17b1 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:08.823279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 771/1527] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:13.139183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005998e9-1f26-40c1-af14-3ceb617c63c5 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:13.139775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005998e9-1f26-40c1-af14-3ceb617c63c5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:13.139893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005998e9-1f26-40c1-af14-3ceb617c63c5 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:13.141355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:41:13.147445 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbcce4-7896-49a8-9f9c-609c96f3e50e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:13.147871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbcce4-7896-49a8-9f9c-609c96f3e50e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:13.148011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbcce4-7896-49a8-9f9c-609c96f3e50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:13.167667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0adbcce4-7896-49a8-9f9c-609c96f3e50e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:13.168901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0adbcce4-7896-49a8-9f9c-609c96f3e50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:13.180826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:13.473901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005998e9-1f26-40c1-af14-3ceb617c63c5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:13.475666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-005998e9-1f26-40c1-af14-3ceb617c63c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:13.551889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005998e9-1f26-40c1-af14-3ceb617c63c5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:13.555440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b57f77-9dbe-46db-bcb2-da12dc3ab140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:13.556035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57f77-9dbe-46db-bcb2-da12dc3ab140 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:13.556316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57f77-9dbe-46db-bcb2-da12dc3ab140 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:13.556615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57f77-9dbe-46db-bcb2-da12dc3ab140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:13.566151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b57f77-9dbe-46db-bcb2-da12dc3ab140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:13.567576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b57f77-9dbe-46db-bcb2-da12dc3ab140 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:13.576679 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 758/1530] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:15.461706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f501581-67b6-4582-b36b-270a35da442f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:15.462281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f501581-67b6-4582-b36b-270a35da442f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:15.462281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f501581-67b6-4582-b36b-270a35da442f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:15.463613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:41:15.468775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e2c1b-bca1-4fac-a745-c69c48f26a00 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:15.469158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e2c1b-bca1-4fac-a745-c69c48f26a00 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:15.469330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e2c1b-bca1-4fac-a745-c69c48f26a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:15.489034 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-430e2c1b-bca1-4fac-a745-c69c48f26a00 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:15.490224 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-430e2c1b-bca1-4fac-a745-c69c48f26a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:15.497915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:15.786967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f501581-67b6-4582-b36b-270a35da442f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:15.797827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f501581-67b6-4582-b36b-270a35da442f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:15.801674 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fda0b-d20d-4cc5-9dcd-9b6255474c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:15.802280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fda0b-d20d-4cc5-9dcd-9b6255474c90 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:15.802562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fda0b-d20d-4cc5-9dcd-9b6255474c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:15.802835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fda0b-d20d-4cc5-9dcd-9b6255474c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:15.811710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fda0b-d20d-4cc5-9dcd-9b6255474c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:15.813170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fda0b-d20d-4cc5-9dcd-9b6255474c90 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:15.823252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 774/1533] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:19.493093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4a6ac-ee5b-499a-983c-e872af0af635 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:19.493620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4a6ac-ee5b-499a-983c-e872af0af635 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:19.493749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4a6ac-ee5b-499a-983c-e872af0af635 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:19.495018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:41:19.500120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9061b-6b7b-4ed5-9cb5-4e362d946918 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:19.500523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9061b-6b7b-4ed5-9cb5-4e362d946918 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:19.500708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9061b-6b7b-4ed5-9cb5-4e362d946918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:19.521122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-45a9061b-6b7b-4ed5-9cb5-4e362d946918 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:19.522292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-45a9061b-6b7b-4ed5-9cb5-4e362d946918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:19.529971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:20.113958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4a6ac-ee5b-499a-983c-e872af0af635 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:20.115624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0dd4a6ac-ee5b-499a-983c-e872af0af635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:20.184480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:20.199533 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4a6ac-ee5b-499a-983c-e872af0af635 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:20.203042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5378ae-3db4-43cc-9e9a-c29ebdf7bd3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:20.203647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5378ae-3db4-43cc-9e9a-c29ebdf7bd3f service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:20.203927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5378ae-3db4-43cc-9e9a-c29ebdf7bd3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:20.204200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5378ae-3db4-43cc-9e9a-c29ebdf7bd3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:20.216183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5378ae-3db4-43cc-9e9a-c29ebdf7bd3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:20.217783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5378ae-3db4-43cc-9e9a-c29ebdf7bd3f service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:20.228035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 761/1536] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:22.325772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46776d-6eb2-4bbf-89a9-27a2c913f2b3 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:22.326330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46776d-6eb2-4bbf-89a9-27a2c913f2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:22.326330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46776d-6eb2-4bbf-89a9-27a2c913f2b3 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:22.327706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:41:22.333285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c697d5-7d68-4eab-a86d-fa0f252fe413 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:22.333715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c697d5-7d68-4eab-a86d-fa0f252fe413 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:22.333840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c697d5-7d68-4eab-a86d-fa0f252fe413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:22.354043 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-62c697d5-7d68-4eab-a86d-fa0f252fe413 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:22.355173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-62c697d5-7d68-4eab-a86d-fa0f252fe413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:22.363792 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:22.925443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46776d-6eb2-4bbf-89a9-27a2c913f2b3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:22.927138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1f46776d-6eb2-4bbf-89a9-27a2c913f2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:22.934727 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:22.941673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46776d-6eb2-4bbf-89a9-27a2c913f2b3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:22.945124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73eb45-6a8c-44f6-949a-0e51a4368539 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:22.945710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73eb45-6a8c-44f6-949a-0e51a4368539 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:22.945989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73eb45-6a8c-44f6-949a-0e51a4368539 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:22.946258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73eb45-6a8c-44f6-949a-0e51a4368539 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:22.955389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73eb45-6a8c-44f6-949a-0e51a4368539 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:22.956857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73eb45-6a8c-44f6-949a-0e51a4368539 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:22.966239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 777/1539] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:32.349196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=11 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:32.962186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:49.195013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd780e-4e70-4a72-adb8-6294a473596d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.199071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd780e-4e70-4a72-adb8-6294a473596d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.199071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd780e-4e70-4a72-adb8-6294a473596d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.217860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33f880-b776-41b4-81f7-b8740d6e4eef None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.218952 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33f880-b776-41b4-81f7-b8740d6e4eef None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.219719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33f880-b776-41b4-81f7-b8740d6e4eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.221619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4ccd780e-4e70-4a72-adb8-6294a473596d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.223019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4ccd780e-4e70-4a72-adb8-6294a473596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.237992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 763/1540] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.251489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdf8b6-4fb2-45e0-89b2-cf629ab06ad0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.251809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdf8b6-4fb2-45e0-89b2-cf629ab06ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.252052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdf8b6-4fb2-45e0-89b2-cf629ab06ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.257732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ef33f880-b776-41b4-81f7-b8740d6e4eef None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.258985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ef33f880-b776-41b4-81f7-b8740d6e4eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.272645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 778/1541] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.277793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8862629-9b2a-4ffc-ba23-0d5bcaef5f67 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.278107 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8862629-9b2a-4ffc-ba23-0d5bcaef5f67 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.278352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8862629-9b2a-4ffc-ba23-0d5bcaef5f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.295930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-18bdf8b6-4fb2-45e0-89b2-cf629ab06ad0 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.297325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-18bdf8b6-4fb2-45e0-89b2-cf629ab06ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.312781 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b8862629-9b2a-4ffc-ba23-0d5bcaef5f67 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.314406 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b8862629-9b2a-4ffc-ba23-0d5bcaef5f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.317310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 764/1542] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.321975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdf8b6-4fb2-45e0-89b2-cf629ab06ad0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:49.334354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdf8b6-4fb2-45e0-89b2-cf629ab06ad0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:49.336545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 779/1543] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:42:49.338096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e123e-2cea-4b77-88d5-5560cd8805b5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.338619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e123e-2cea-4b77-88d5-5560cd8805b5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.339120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e123e-2cea-4b77-88d5-5560cd8805b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.353924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ea67f-9ba8-454a-b55c-540d75b9b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:49.354559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ea67f-9ba8-454a-b55c-540d75b9b7ff admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.354840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ea67f-9ba8-454a-b55c-540d75b9b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.355114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ea67f-9ba8-454a-b55c-540d75b9b7ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.356392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ea67f-9ba8-454a-b55c-540d75b9b7ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:42:49.356392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ea67f-9ba8-454a-b55c-540d75b9b7ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:49.362120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f22e123e-2cea-4b77-88d5-5560cd8805b5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.363674 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f22e123e-2cea-4b77-88d5-5560cd8805b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.366286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ea67f-9ba8-454a-b55c-540d75b9b7ff admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:49.379889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 765/1544] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:42:49.384742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ea67f-9ba8-454a-b55c-540d75b9b7ff admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:49.385098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 780/1545] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.395954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ea67f-9ba8-454a-b55c-540d75b9b7ff admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:49.405648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e123e-2cea-4b77-88d5-5560cd8805b5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:49.425396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e123e-2cea-4b77-88d5-5560cd8805b5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:49.426235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5619a95-91b9-4c15-a774-89461b8a45d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:49.428871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5619a95-91b9-4c15-a774-89461b8a45d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.429643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5619a95-91b9-4c15-a774-89461b8a45d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.429643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5619a95-91b9-4c15-a774-89461b8a45d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.430178 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5619a95-91b9-4c15-a774-89461b8a45d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:42:49.430491 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5619a95-91b9-4c15-a774-89461b8a45d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:49.432355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5619a95-91b9-4c15-a774-89461b8a45d2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:49.436224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141274e-bd68-4eb8-981e-1121339bc15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:49.437127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141274e-bd68-4eb8-981e-1121339bc15a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.437626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141274e-bd68-4eb8-981e-1121339bc15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.438123 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141274e-bd68-4eb8-981e-1121339bc15a admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.438995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141274e-bd68-4eb8-981e-1121339bc15a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:49.441436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141274e-bd68-4eb8-981e-1121339bc15a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:49.461036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 766/1546] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:42:49.462361 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ecb54-1f29-4db4-975b-cea2522f0fdc None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.462710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ecb54-1f29-4db4-975b-cea2522f0fdc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.462983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ecb54-1f29-4db4-975b-cea2522f0fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.496391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-933ecb54-1f29-4db4-975b-cea2522f0fdc None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.498248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-933ecb54-1f29-4db4-975b-cea2522f0fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.512183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 767/1547] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.513484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624401d-cb18-409b-8489-d15fdcdd9d7d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.513769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624401d-cb18-409b-8489-d15fdcdd9d7d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.514041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624401d-cb18-409b-8489-d15fdcdd9d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.518111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 781/1548] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:49.518942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141274e-bd68-4eb8-981e-1121339bc15a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:49.551662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c624401d-cb18-409b-8489-d15fdcdd9d7d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.553357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c624401d-cb18-409b-8489-d15fdcdd9d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.608599 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141274e-bd68-4eb8-981e-1121339bc15a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:49.614010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493fa10-64b8-43a6-81a3-ea08b796c974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:49.614683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493fa10-64b8-43a6-81a3-ea08b796c974 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.614963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493fa10-64b8-43a6-81a3-ea08b796c974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.615241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493fa10-64b8-43a6-81a3-ea08b796c974 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.616006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493fa10-64b8-43a6-81a3-ea08b796c974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:49.617564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 768/1549] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.618018 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493fa10-64b8-43a6-81a3-ea08b796c974 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:49.618829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ec8c7-3f33-441c-9d2f-5a9d861e0bc0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.619122 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ec8c7-3f33-441c-9d2f-5a9d861e0bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.619397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ec8c7-3f33-441c-9d2f-5a9d861e0bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.642538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b03ec8c7-3f33-441c-9d2f-5a9d861e0bc0 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.644127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b03ec8c7-3f33-441c-9d2f-5a9d861e0bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.647768 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 782/1550] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:49.648716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493fa10-64b8-43a6-81a3-ea08b796c974 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:49.709610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 769/1551] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.710262 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ec8c7-3f33-441c-9d2f-5a9d861e0bc0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:49.720597 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ec8c7-3f33-441c-9d2f-5a9d861e0bc0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:49.721156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493fa10-64b8-43a6-81a3-ea08b796c974 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:49.725999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a3d77-2145-441c-848c-ba07fa6a00ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:49.726607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a3d77-2145-441c-848c-ba07fa6a00ae admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.726809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8dcab1-3c9d-4005-b4b7-7e2542e5c597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:49.727018 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a3d77-2145-441c-848c-ba07fa6a00ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.727200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a3d77-2145-441c-848c-ba07fa6a00ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.727311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dcab1-3c9d-4005-b4b7-7e2542e5c597 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.727515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dcab1-3c9d-4005-b4b7-7e2542e5c597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.727787 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dcab1-3c9d-4005-b4b7-7e2542e5c597 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.727909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a3d77-2145-441c-848c-ba07fa6a00ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:42:49.728088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a3d77-2145-441c-848c-ba07fa6a00ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:49.728414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8dcab1-3c9d-4005-b4b7-7e2542e5c597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:49.729126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-307a3d77-2145-441c-848c-ba07fa6a00ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:42:49.729126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 31 11:42:49.729126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: Aug 31 11:42:49.730011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a3d77-2145-441c-848c-ba07fa6a00ae admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:49.730385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8dcab1-3c9d-4005-b4b7-7e2542e5c597 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:49.747776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 783/1552] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:42:49.748971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927180c-7dbd-402f-b0fa-2dcb627666e2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.749254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927180c-7dbd-402f-b0fa-2dcb627666e2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.749528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927180c-7dbd-402f-b0fa-2dcb627666e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.760639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 770/1553] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/users => generated 284 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:49.761899 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b76d99-0df4-4eaf-93f3-bea661a3ec02 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.762191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b76d99-0df4-4eaf-93f3-bea661a3ec02 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.762468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b76d99-0df4-4eaf-93f3-bea661a3ec02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.771040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f927180c-7dbd-402f-b0fa-2dcb627666e2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.772237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f927180c-7dbd-402f-b0fa-2dcb627666e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.780299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 784/1554] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.781505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ce2b7-8b2c-4637-8d8a-e52bf8a7ee92 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.781787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ce2b7-8b2c-4637-8d8a-e52bf8a7ee92 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.782053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ce2b7-8b2c-4637-8d8a-e52bf8a7ee92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.782583 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-44b76d99-0df4-4eaf-93f3-bea661a3ec02 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.783786 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-44b76d99-0df4-4eaf-93f3-bea661a3ec02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.792373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 771/1555] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.793632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100317bf-d9d5-4551-89f7-528ebedba659 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.793910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100317bf-d9d5-4551-89f7-528ebedba659 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.794186 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100317bf-d9d5-4551-89f7-528ebedba659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.805592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e69ce2b7-8b2c-4637-8d8a-e52bf8a7ee92 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.806782 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e69ce2b7-8b2c-4637-8d8a-e52bf8a7ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.813637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-100317bf-d9d5-4551-89f7-528ebedba659 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.814616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 785/1556] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.814810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-100317bf-d9d5-4551-89f7-528ebedba659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.816844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ce2b7-8b2c-4637-8d8a-e52bf8a7ee92 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:49.824143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 772/1557] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.825333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c3c66-ad26-45cf-80c8-7b0a28d8f4e1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.825714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c3c66-ad26-45cf-80c8-7b0a28d8f4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.826061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c3c66-ad26-45cf-80c8-7b0a28d8f4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.827444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ce2b7-8b2c-4637-8d8a-e52bf8a7ee92 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:49.832094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25942cf-1564-4d2d-aea3-c35cacbe0b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:49.833006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25942cf-1564-4d2d-aea3-c35cacbe0b29 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.833330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25942cf-1564-4d2d-aea3-c35cacbe0b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.833814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25942cf-1564-4d2d-aea3-c35cacbe0b29 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.835332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25942cf-1564-4d2d-aea3-c35cacbe0b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:49.836356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-e25942cf-1564-4d2d-aea3-c35cacbe0b29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:42:49.836356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 31 11:42:49.836356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: Aug 31 11:42:49.837611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25942cf-1564-4d2d-aea3-c35cacbe0b29 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:49.845292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a78c3c66-ad26-45cf-80c8-7b0a28d8f4e1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.846500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a78c3c66-ad26-45cf-80c8-7b0a28d8f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.855522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:49.858209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 773/1558] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.859118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c3c66-ad26-45cf-80c8-7b0a28d8f4e1 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:49.862853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 786/1559] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/users => generated 293 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:49.864403 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25942cf-1564-4d2d-aea3-c35cacbe0b29 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:49.871202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c3c66-ad26-45cf-80c8-7b0a28d8f4e1 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:49.875270 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25942cf-1564-4d2d-aea3-c35cacbe0b29 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:49.876363 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dedd3-adae-49d5-9d22-e54c0f018ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:49.877100 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dedd3-adae-49d5-9d22-e54c0f018ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.877386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dedd3-adae-49d5-9d22-e54c0f018ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.877663 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dedd3-adae-49d5-9d22-e54c0f018ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.878293 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dedd3-adae-49d5-9d22-e54c0f018ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:49.879359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-389dedd3-adae-49d5-9d22-e54c0f018ee4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:42:49.879359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 31 11:42:49.879359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: Aug 31 11:42:49.880263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d4e68-00b2-4a3e-9b2a-200aa6a5464b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:49.880726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dedd3-adae-49d5-9d22-e54c0f018ee4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:49.880891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d4e68-00b2-4a3e-9b2a-200aa6a5464b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.881172 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d4e68-00b2-4a3e-9b2a-200aa6a5464b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.881443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d4e68-00b2-4a3e-9b2a-200aa6a5464b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.882111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d4e68-00b2-4a3e-9b2a-200aa6a5464b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:49.883696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d4e68-00b2-4a3e-9b2a-200aa6a5464b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:49.893969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 787/1560] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:49.895346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b9470-777c-41aa-a321-a31e870b8934 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.895629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b9470-777c-41aa-a321-a31e870b8934 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.895904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b9470-777c-41aa-a321-a31e870b8934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.898197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:49.915667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 774/1561] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:49.915667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea85115-7ee4-4be2-840c-504fdb0383d4 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.915667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea85115-7ee4-4be2-840c-504fdb0383d4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.915667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea85115-7ee4-4be2-840c-504fdb0383d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.917670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b44b9470-777c-41aa-a321-a31e870b8934 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.919016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b44b9470-777c-41aa-a321-a31e870b8934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.933904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cea85115-7ee4-4be2-840c-504fdb0383d4 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:49.935082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cea85115-7ee4-4be2-840c-504fdb0383d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:49.979808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 788/1562] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.981173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8458cf-aff3-41b7-9ce8-b29428a73309 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:49.981524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8458cf-aff3-41b7-9ce8-b29428a73309 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:49.981873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8458cf-aff3-41b7-9ce8-b29428a73309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:49.994243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 775/1563] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:49.995074 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea85115-7ee4-4be2-840c-504fdb0383d4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.001607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3c8458cf-aff3-41b7-9ce8-b29428a73309 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:50.002815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3c8458cf-aff3-41b7-9ce8-b29428a73309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:50.003999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea85115-7ee4-4be2-840c-504fdb0383d4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.009224 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbce18-c8ec-42a8-88b7-dc7a29dd368f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.009815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbce18-c8ec-42a8-88b7-dc7a29dd368f admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.010095 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbce18-c8ec-42a8-88b7-dc7a29dd368f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.010364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbce18-c8ec-42a8-88b7-dc7a29dd368f admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.011002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbce18-c8ec-42a8-88b7-dc7a29dd368f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:42:50.011310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 789/1564] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:50.011430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbce18-c8ec-42a8-88b7-dc7a29dd368f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.012996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbce18-c8ec-42a8-88b7-dc7a29dd368f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.013492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8458cf-aff3-41b7-9ce8-b29428a73309 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.024341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8458cf-aff3-41b7-9ce8-b29428a73309 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.031846 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 776/1565] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:42:50.031846 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbce18-c8ec-42a8-88b7-dc7a29dd368f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.038383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c8d3b-5300-4bc0-8d6f-c7f275544f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.038629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbce18-c8ec-42a8-88b7-dc7a29dd368f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.038999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c8d3b-5300-4bc0-8d6f-c7f275544f50 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.039279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c8d3b-5300-4bc0-8d6f-c7f275544f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.039582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c8d3b-5300-4bc0-8d6f-c7f275544f50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.040262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c8d3b-5300-4bc0-8d6f-c7f275544f50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:42:50.040596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c8d3b-5300-4bc0-8d6f-c7f275544f50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.042192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c8d3b-5300-4bc0-8d6f-c7f275544f50 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.043584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fc72c-812c-48af-86d2-6c66406ac744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.044207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fc72c-812c-48af-86d2-6c66406ac744 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.044515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fc72c-812c-48af-86d2-6c66406ac744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.044797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fc72c-812c-48af-86d2-6c66406ac744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.045444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fc72c-812c-48af-86d2-6c66406ac744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.047055 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fc72c-812c-48af-86d2-6c66406ac744 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.054438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 790/1566] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:42:50.055315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c8d3b-5300-4bc0-8d6f-c7f275544f50 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.057968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 777/1567] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:50.058821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fc72c-812c-48af-86d2-6c66406ac744 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.068607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c8d3b-5300-4bc0-8d6f-c7f275544f50 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.069546 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fc72c-812c-48af-86d2-6c66406ac744 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.073355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffe578-44cc-451b-bacb-f1461d48c124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.073939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe578-44cc-451b-bacb-f1461d48c124 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.074196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12fb24-d5ae-4d92-a060-cd04c9d8bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.074345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe578-44cc-451b-bacb-f1461d48c124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.074461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe578-44cc-451b-bacb-f1461d48c124 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.074787 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12fb24-d5ae-4d92-a060-cd04c9d8bbe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.075072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12fb24-d5ae-4d92-a060-cd04c9d8bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.075173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffe578-44cc-451b-bacb-f1461d48c124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.075349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12fb24-d5ae-4d92-a060-cd04c9d8bbe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.075970 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12fb24-d5ae-4d92-a060-cd04c9d8bbe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.076698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffe578-44cc-451b-bacb-f1461d48c124 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.077907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12fb24-d5ae-4d92-a060-cd04c9d8bbe9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.090222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 791/1568] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:50.090222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffe578-44cc-451b-bacb-f1461d48c124 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.098322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffe578-44cc-451b-bacb-f1461d48c124 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.105822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 778/1569] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/users => generated 302 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:50.105983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71658e-1b67-4e70-a120-a6c451466d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.106098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12fb24-d5ae-4d92-a060-cd04c9d8bbe9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.106204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71658e-1b67-4e70-a120-a6c451466d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.106204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71658e-1b67-4e70-a120-a6c451466d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.106204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71658e-1b67-4e70-a120-a6c451466d4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.106204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71658e-1b67-4e70-a120-a6c451466d4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:42:50.106204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71658e-1b67-4e70-a120-a6c451466d4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.107319 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71658e-1b67-4e70-a120-a6c451466d4a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.113216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12fb24-d5ae-4d92-a060-cd04c9d8bbe9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.118215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d0fb1-c201-483b-94e5-41c6cb85195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.118832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d0fb1-c201-483b-94e5-41c6cb85195f admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.119236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d0fb1-c201-483b-94e5-41c6cb85195f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.119421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 792/1570] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:42:50.126683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d0fb1-c201-483b-94e5-41c6cb85195f admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.126683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d0fb1-c201-483b-94e5-41c6cb85195f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:42:50.126844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71658e-1b67-4e70-a120-a6c451466d4a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.126919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d0fb1-c201-483b-94e5-41c6cb85195f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.126919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d0fb1-c201-483b-94e5-41c6cb85195f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.130644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71658e-1b67-4e70-a120-a6c451466d4a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.136518 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ce27d-cee8-47c9-bef9-5221760c99d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.137181 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ce27d-cee8-47c9-bef9-5221760c99d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.137492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ce27d-cee8-47c9-bef9-5221760c99d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.137769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ce27d-cee8-47c9-bef9-5221760c99d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.138407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ce27d-cee8-47c9-bef9-5221760c99d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:42:50.138713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ce27d-cee8-47c9-bef9-5221760c99d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.139163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 779/1571] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:42:50.139972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d0fb1-c201-483b-94e5-41c6cb85195f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.140461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ce27d-cee8-47c9-bef9-5221760c99d5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.152940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 793/1572] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:42:50.153836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ce27d-cee8-47c9-bef9-5221760c99d5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.155595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d0fb1-c201-483b-94e5-41c6cb85195f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.161148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021157a6-314b-4f4f-b0bc-fdb86807a38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.161757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021157a6-314b-4f4f-b0bc-fdb86807a38c admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.162037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021157a6-314b-4f4f-b0bc-fdb86807a38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.162312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021157a6-314b-4f4f-b0bc-fdb86807a38c admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.162941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021157a6-314b-4f4f-b0bc-fdb86807a38c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.164737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ce27d-cee8-47c9-bef9-5221760c99d5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.164896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021157a6-314b-4f4f-b0bc-fdb86807a38c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.169332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fa375-fdb3-4e27-bc7e-187fa9ee1fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.169838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fa375-fdb3-4e27-bc7e-187fa9ee1fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.170150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fa375-fdb3-4e27-bc7e-187fa9ee1fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.170428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fa375-fdb3-4e27-bc7e-187fa9ee1fe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.171055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fa375-fdb3-4e27-bc7e-187fa9ee1fe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.172975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fa375-fdb3-4e27-bc7e-187fa9ee1fe8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.193948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 780/1573] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:50.194849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021157a6-314b-4f4f-b0bc-fdb86807a38c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.204607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 794/1574] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:50.205605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fa375-fdb3-4e27-bc7e-187fa9ee1fe8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.256943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021157a6-314b-4f4f-b0bc-fdb86807a38c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.271793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fa375-fdb3-4e27-bc7e-187fa9ee1fe8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.276327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ccdb3-f3a7-4134-9b11-9651e314f840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.277014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ccdb3-f3a7-4134-9b11-9651e314f840 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.277291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ccdb3-f3a7-4134-9b11-9651e314f840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.277562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ccdb3-f3a7-4134-9b11-9651e314f840 admin admin] PATH_INFO: `/v3/projects/283f52c7378b41a79ad6803f87b19a07/users/2b9465185a57462f823a40711d8c3742/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.282785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:50.289397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:50.292580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ccdb3-f3a7-4134-9b11-9651e314f840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=283f52c7378b41a79ad6803f87b19a07, user_id=2b9465185a57462f823a40711d8c3742, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.295861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ccdb3-f3a7-4134-9b11-9651e314f840 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.310210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 795/1575] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:50 2022] PUT /identity/v3/projects/283f52c7378b41a79ad6803f87b19a07/users/2b9465185a57462f823a40711d8c3742/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:50.310845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ccdb3-f3a7-4134-9b11-9651e314f840 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.319759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59992d0-ca3b-4461-9d14-af083331b3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.320384 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59992d0-ca3b-4461-9d14-af083331b3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.321003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59992d0-ca3b-4461-9d14-af083331b3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.321374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59992d0-ca3b-4461-9d14-af083331b3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.322168 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59992d0-ca3b-4461-9d14-af083331b3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.323799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59992d0-ca3b-4461-9d14-af083331b3ad admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.335096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 781/1576] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:50.336080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59992d0-ca3b-4461-9d14-af083331b3ad admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.347780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59992d0-ca3b-4461-9d14-af083331b3ad admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.380438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ccdb3-f3a7-4134-9b11-9651e314f840 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.385297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9cf8b-f9ea-49a2-a5bf-ad0a514c6c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.385901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9cf8b-f9ea-49a2-a5bf-ad0a514c6c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.386182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9cf8b-f9ea-49a2-a5bf-ad0a514c6c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.386456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9cf8b-f9ea-49a2-a5bf-ad0a514c6c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.387118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9cf8b-f9ea-49a2-a5bf-ad0a514c6c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.388754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9cf8b-f9ea-49a2-a5bf-ad0a514c6c60 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.399207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 796/1577] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:50.402237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9cf8b-f9ea-49a2-a5bf-ad0a514c6c60 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.404645 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dec904-d57f-42dc-9e74-a8575f9b5c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.405264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dec904-d57f-42dc-9e74-a8575f9b5c69 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.405530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dec904-d57f-42dc-9e74-a8575f9b5c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.405802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dec904-d57f-42dc-9e74-a8575f9b5c69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.406437 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dec904-d57f-42dc-9e74-a8575f9b5c69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.408325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dec904-d57f-42dc-9e74-a8575f9b5c69 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.409736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9cf8b-f9ea-49a2-a5bf-ad0a514c6c60 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.415249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3006d8-70d1-4ea2-b96d-82db2a13c931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.415831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3006d8-70d1-4ea2-b96d-82db2a13c931 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.416113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3006d8-70d1-4ea2-b96d-82db2a13c931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.416384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3006d8-70d1-4ea2-b96d-82db2a13c931 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.417037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3006d8-70d1-4ea2-b96d-82db2a13c931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.418971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3006d8-70d1-4ea2-b96d-82db2a13c931 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.441322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 782/1578] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:50.441322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dec904-d57f-42dc-9e74-a8575f9b5c69 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.448808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 797/1579] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:50.449665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3006d8-70d1-4ea2-b96d-82db2a13c931 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.507797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dec904-d57f-42dc-9e74-a8575f9b5c69 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.511923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3006d8-70d1-4ea2-b96d-82db2a13c931 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.512736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036f3dd-702b-4955-9975-36a5279992f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.513340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036f3dd-702b-4955-9975-36a5279992f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.513620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036f3dd-702b-4955-9975-36a5279992f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.513888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036f3dd-702b-4955-9975-36a5279992f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.514520 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036f3dd-702b-4955-9975-36a5279992f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.516420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036f3dd-702b-4955-9975-36a5279992f2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.516635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b67328-86cb-4e11-bfdb-f341e86f31b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.517238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b67328-86cb-4e11-bfdb-f341e86f31b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.517523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b67328-86cb-4e11-bfdb-f341e86f31b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.517799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b67328-86cb-4e11-bfdb-f341e86f31b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.518421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b67328-86cb-4e11-bfdb-f341e86f31b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.520606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b67328-86cb-4e11-bfdb-f341e86f31b7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.545462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 798/1580] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:50.545462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b67328-86cb-4e11-bfdb-f341e86f31b7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.547962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 783/1581] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:50.549431 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afbd05-a5f1-4c80-bc9e-96aa917e4c98 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.549790 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afbd05-a5f1-4c80-bc9e-96aa917e4c98 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.550682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afbd05-a5f1-4c80-bc9e-96aa917e4c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.554385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b67328-86cb-4e11-bfdb-f341e86f31b7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.580562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d0afbd05-a5f1-4c80-bc9e-96aa917e4c98 None None] MFA Rules not processed for user `2b9465185a57462f823a40711d8c3742`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:50.581790 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d0afbd05-a5f1-4c80-bc9e-96aa917e4c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:50.603625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:50.607246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:50.616333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e933dbb-41f8-44f5-a908-1c5480b4efbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.617148 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e933dbb-41f8-44f5-a908-1c5480b4efbe admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.617426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e933dbb-41f8-44f5-a908-1c5480b4efbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.617693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e933dbb-41f8-44f5-a908-1c5480b4efbe admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.618373 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e933dbb-41f8-44f5-a908-1c5480b4efbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.620332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e933dbb-41f8-44f5-a908-1c5480b4efbe admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.644093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 799/1582] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:50.644998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e933dbb-41f8-44f5-a908-1c5480b4efbe admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.651502 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 784/1583] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:50.652451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afbd05-a5f1-4c80-bc9e-96aa917e4c98 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.654407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e933dbb-41f8-44f5-a908-1c5480b4efbe admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.661834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8cb47-4192-4ec2-9316-b47587aeb7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.662587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8cb47-4192-4ec2-9316-b47587aeb7bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.662949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8cb47-4192-4ec2-9316-b47587aeb7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.663304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8cb47-4192-4ec2-9316-b47587aeb7bf admin admin] PATH_INFO: `/v3/projects/b4ac4dc41f7e420d811483990e020f22/users/5de832888ffd4a69ae88cac6b632304e/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.663989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afbd05-a5f1-4c80-bc9e-96aa917e4c98 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.668987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387e021-1c44-4b81-bdd3-86a0fafe54f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.669592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387e021-1c44-4b81-bdd3-86a0fafe54f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.669876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387e021-1c44-4b81-bdd3-86a0fafe54f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.670149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387e021-1c44-4b81-bdd3-86a0fafe54f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.670793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387e021-1c44-4b81-bdd3-86a0fafe54f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.672417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387e021-1c44-4b81-bdd3-86a0fafe54f4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.682578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8cb47-4192-4ec2-9316-b47587aeb7bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ac4dc41f7e420d811483990e020f22, user_id=5de832888ffd4a69ae88cac6b632304e, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.683641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 785/1584] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:50.684474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387e021-1c44-4b81-bdd3-86a0fafe54f4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.688645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8cb47-4192-4ec2-9316-b47587aeb7bf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.694778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387e021-1c44-4b81-bdd3-86a0fafe54f4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.699766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba824e6-b3b8-4f91-8f90-e2dc760fb329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.700375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba824e6-b3b8-4f91-8f90-e2dc760fb329 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.700703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba824e6-b3b8-4f91-8f90-e2dc760fb329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.700974 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba824e6-b3b8-4f91-8f90-e2dc760fb329 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.701594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba824e6-b3b8-4f91-8f90-e2dc760fb329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.703237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 800/1585] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:50 2022] PUT /identity/v3/projects/b4ac4dc41f7e420d811483990e020f22/users/5de832888ffd4a69ae88cac6b632304e/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:50.703653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba824e6-b3b8-4f91-8f90-e2dc760fb329 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.705428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8cb47-4192-4ec2-9316-b47587aeb7bf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.730042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 786/1586] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:50.730042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba824e6-b3b8-4f91-8f90-e2dc760fb329 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.737829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba824e6-b3b8-4f91-8f90-e2dc760fb329 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.770940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8cb47-4192-4ec2-9316-b47587aeb7bf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.775583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b56af-0e60-4bf5-a353-13885aa29489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.776186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b56af-0e60-4bf5-a353-13885aa29489 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.776469 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b56af-0e60-4bf5-a353-13885aa29489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.776771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b56af-0e60-4bf5-a353-13885aa29489 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.777403 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b56af-0e60-4bf5-a353-13885aa29489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.779338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b56af-0e60-4bf5-a353-13885aa29489 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.795912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4475cf-8a3b-4fa3-9d1b-aced756138f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.796550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4475cf-8a3b-4fa3-9d1b-aced756138f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.796840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4475cf-8a3b-4fa3-9d1b-aced756138f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.797114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4475cf-8a3b-4fa3-9d1b-aced756138f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.797764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4475cf-8a3b-4fa3-9d1b-aced756138f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.799370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4475cf-8a3b-4fa3-9d1b-aced756138f0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.802346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 801/1587] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:50.803173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b56af-0e60-4bf5-a353-13885aa29489 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.810817 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 787/1588] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:50.811611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4475cf-8a3b-4fa3-9d1b-aced756138f0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.816149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b56af-0e60-4bf5-a353-13885aa29489 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.821564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b2e51-d0d9-4b51-8ee1-d9b606f752ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.822240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b2e51-d0d9-4b51-8ee1-d9b606f752ea admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.822474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4475cf-8a3b-4fa3-9d1b-aced756138f0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.822696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b2e51-d0d9-4b51-8ee1-d9b606f752ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.823031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b2e51-d0d9-4b51-8ee1-d9b606f752ea admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.823675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b2e51-d0d9-4b51-8ee1-d9b606f752ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.825655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b2e51-d0d9-4b51-8ee1-d9b606f752ea admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.827218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8fe97-6e5a-4aaf-9473-12400b41e350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.827814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8fe97-6e5a-4aaf-9473-12400b41e350 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.828101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8fe97-6e5a-4aaf-9473-12400b41e350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.828374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8fe97-6e5a-4aaf-9473-12400b41e350 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.829961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8fe97-6e5a-4aaf-9473-12400b41e350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.831962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8fe97-6e5a-4aaf-9473-12400b41e350 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.844126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 788/1589] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:50.845378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8fe97-6e5a-4aaf-9473-12400b41e350 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.851914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 802/1590] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:50.852750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b2e51-d0d9-4b51-8ee1-d9b606f752ea admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.862103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b2e51-d0d9-4b51-8ee1-d9b606f752ea admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.867397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55dab4a-c1b4-40f2-b4c3-e4a9a7adfac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.868041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dab4a-c1b4-40f2-b4c3-e4a9a7adfac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.868323 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dab4a-c1b4-40f2-b4c3-e4a9a7adfac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.868654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dab4a-c1b4-40f2-b4c3-e4a9a7adfac3 admin admin] PATH_INFO: `/v3/projects/3201370921ff48d292b0cb81fa97d84f/users/78ce88f533c04055bcc81f39d31da988/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.884561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55dab4a-c1b4-40f2-b4c3-e4a9a7adfac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3201370921ff48d292b0cb81fa97d84f, user_id=78ce88f533c04055bcc81f39d31da988, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:50.887275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55dab4a-c1b4-40f2-b4c3-e4a9a7adfac3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:50.901414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 803/1591] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:50 2022] PUT /identity/v3/projects/3201370921ff48d292b0cb81fa97d84f/users/78ce88f533c04055bcc81f39d31da988/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:50.902097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8fe97-6e5a-4aaf-9473-12400b41e350 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.902636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d659d49-6c96-4635-97ff-1669cd448ee2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.902920 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d659d49-6c96-4635-97ff-1669cd448ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.903190 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d659d49-6c96-4635-97ff-1669cd448ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.922586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2d659d49-6c96-4635-97ff-1669cd448ee2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:50.923794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2d659d49-6c96-4635-97ff-1669cd448ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:50.940761 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:50.959317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5e937-c082-4477-b4d3-b23233cd93b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.959923 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5e937-c082-4477-b4d3-b23233cd93b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.960207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5e937-c082-4477-b4d3-b23233cd93b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.960482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5e937-c082-4477-b4d3-b23233cd93b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:50.982854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 804/1592] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:50.983698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d659d49-6c96-4635-97ff-1669cd448ee2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:50.993588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d659d49-6c96-4635-97ff-1669cd448ee2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:50.998523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38136cfa-6400-420c-8691-02e03e36d68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:50.999150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38136cfa-6400-420c-8691-02e03e36d68a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:50.999433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38136cfa-6400-420c-8691-02e03e36d68a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:50.999706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38136cfa-6400-420c-8691-02e03e36d68a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.000343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38136cfa-6400-420c-8691-02e03e36d68a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.001968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38136cfa-6400-420c-8691-02e03e36d68a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.004799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:51.014028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 805/1593] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:51.015120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38136cfa-6400-420c-8691-02e03e36d68a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.024732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38136cfa-6400-420c-8691-02e03e36d68a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.026056 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5e937-c082-4477-b4d3-b23233cd93b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.027503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5e937-c082-4477-b4d3-b23233cd93b8 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.029197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeab517-16b3-4b60-bf48-9e900e5f7b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.029771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeab517-16b3-4b60-bf48-9e900e5f7b4b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.030053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeab517-16b3-4b60-bf48-9e900e5f7b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.030322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeab517-16b3-4b60-bf48-9e900e5f7b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.030947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeab517-16b3-4b60-bf48-9e900e5f7b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.032523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeab517-16b3-4b60-bf48-9e900e5f7b4b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.036607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 789/1594] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:51.037414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5e937-c082-4477-b4d3-b23233cd93b8 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.042541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 806/1595] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:51.043411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeab517-16b3-4b60-bf48-9e900e5f7b4b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.047473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5e937-c082-4477-b4d3-b23233cd93b8 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.053373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb06a3e-bb5b-4ff0-9137-53cca2ad0df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.054017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb06a3e-bb5b-4ff0-9137-53cca2ad0df8 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.054324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb06a3e-bb5b-4ff0-9137-53cca2ad0df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.054560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb06a3e-bb5b-4ff0-9137-53cca2ad0df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.055241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb06a3e-bb5b-4ff0-9137-53cca2ad0df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.055373 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeab517-16b3-4b60-bf48-9e900e5f7b4b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.056874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb06a3e-bb5b-4ff0-9137-53cca2ad0df8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.060927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dc1cb-6cc0-4f52-b252-c9a209985c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.061540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dc1cb-6cc0-4f52-b252-c9a209985c3f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.061825 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dc1cb-6cc0-4f52-b252-c9a209985c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.062097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dc1cb-6cc0-4f52-b252-c9a209985c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.062733 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38dc1cb-6cc0-4f52-b252-c9a209985c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.064327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38dc1cb-6cc0-4f52-b252-c9a209985c3f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.068692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 790/1596] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:51.069610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb06a3e-bb5b-4ff0-9137-53cca2ad0df8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.076250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 807/1597] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:51.077719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b945e-7d37-4fe2-a0a9-9d6a0c233391 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.078010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b945e-7d37-4fe2-a0a9-9d6a0c233391 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.078281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b945e-7d37-4fe2-a0a9-9d6a0c233391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.083368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb06a3e-bb5b-4ff0-9137-53cca2ad0df8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.088773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90902f4-5220-46fa-969f-31966761392e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.089403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90902f4-5220-46fa-969f-31966761392e admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.089726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90902f4-5220-46fa-969f-31966761392e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.089997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90902f4-5220-46fa-969f-31966761392e admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.090628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90902f4-5220-46fa-969f-31966761392e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.092215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90902f4-5220-46fa-969f-31966761392e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.096889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-580b945e-7d37-4fe2-a0a9-9d6a0c233391 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:51.098246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-580b945e-7d37-4fe2-a0a9-9d6a0c233391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:51.102340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 791/1598] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:51.103145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90902f4-5220-46fa-969f-31966761392e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.110243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 808/1599] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:51.111286 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b945e-7d37-4fe2-a0a9-9d6a0c233391 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.126276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b945e-7d37-4fe2-a0a9-9d6a0c233391 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.131954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c3d7e-b741-4500-87d1-3085cebd9bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.132561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c3d7e-b741-4500-87d1-3085cebd9bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.132849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c3d7e-b741-4500-87d1-3085cebd9bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.133124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c3d7e-b741-4500-87d1-3085cebd9bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.133759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c3d7e-b741-4500-87d1-3085cebd9bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.135381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c3d7e-b741-4500-87d1-3085cebd9bd8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.146671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 809/1600] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:51.147537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c3d7e-b741-4500-87d1-3085cebd9bd8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.157658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c3d7e-b741-4500-87d1-3085cebd9bd8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.162654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888d40c-7ae5-4b9e-8f35-d62599593265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.163251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888d40c-7ae5-4b9e-8f35-d62599593265 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.163533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888d40c-7ae5-4b9e-8f35-d62599593265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.163804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888d40c-7ae5-4b9e-8f35-d62599593265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.164437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888d40c-7ae5-4b9e-8f35-d62599593265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.166080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888d40c-7ae5-4b9e-8f35-d62599593265 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.167360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90902f4-5220-46fa-969f-31966761392e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.171421 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed8ae4-f312-43c0-9f89-42e52cf5c913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.172777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed8ae4-f312-43c0-9f89-42e52cf5c913 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.172777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed8ae4-f312-43c0-9f89-42e52cf5c913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.172777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed8ae4-f312-43c0-9f89-42e52cf5c913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.176805 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 810/1601] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:51.177490 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888d40c-7ae5-4b9e-8f35-d62599593265 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.186408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed8ae4-f312-43c0-9f89-42e52cf5c913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.187902 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed8ae4-f312-43c0-9f89-42e52cf5c913 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.189086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888d40c-7ae5-4b9e-8f35-d62599593265 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.194900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de12b9-6011-4a28-8a6f-231eb9513ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.195549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de12b9-6011-4a28-8a6f-231eb9513ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.195831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de12b9-6011-4a28-8a6f-231eb9513ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.196106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de12b9-6011-4a28-8a6f-231eb9513ca1 admin admin] PATH_INFO: `/v3/projects/11a0149745754ac59b97e489312dd3ab/users/0d27f92c992749cfa5502babb56cb5a4/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.198609 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 792/1602] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:51.199432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed8ae4-f312-43c0-9f89-42e52cf5c913 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.214697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed8ae4-f312-43c0-9f89-42e52cf5c913 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.217771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de12b9-6011-4a28-8a6f-231eb9513ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11a0149745754ac59b97e489312dd3ab, user_id=0d27f92c992749cfa5502babb56cb5a4, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.221099 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5b89f-cb1f-4374-980f-46f344f7492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.221510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de12b9-6011-4a28-8a6f-231eb9513ca1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.222310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5b89f-cb1f-4374-980f-46f344f7492b admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.223416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5b89f-cb1f-4374-980f-46f344f7492b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.224019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5b89f-cb1f-4374-980f-46f344f7492b admin admin] PATH_INFO: `/v3/projects/aacdfff11f2a41f889f7d7cdc41c12dc/users/38c4d208994a40fe82310adb18bb3547/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.238448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 811/1603] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:51 2022] PUT /identity/v3/projects/11a0149745754ac59b97e489312dd3ab/users/0d27f92c992749cfa5502babb56cb5a4/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:51.239697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de12b9-6011-4a28-8a6f-231eb9513ca1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.249157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5b89f-cb1f-4374-980f-46f344f7492b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aacdfff11f2a41f889f7d7cdc41c12dc, user_id=38c4d208994a40fe82310adb18bb3547, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.250223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-daf5b89f-cb1f-4374-980f-46f344f7492b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:42:51.250223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 31 11:42:51.250223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: Aug 31 11:42:51.252529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5b89f-cb1f-4374-980f-46f344f7492b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.276721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 793/1604] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:51 2022] PUT /identity/v3/projects/aacdfff11f2a41f889f7d7cdc41c12dc/users/38c4d208994a40fe82310adb18bb3547/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:51.281719 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5b89f-cb1f-4374-980f-46f344f7492b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.327263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de12b9-6011-4a28-8a6f-231eb9513ca1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.356347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:51.364054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5b89f-cb1f-4374-980f-46f344f7492b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.369229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd0800-e329-4b65-946e-9c6c5aeebda5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.369984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0800-e329-4b65-946e-9c6c5aeebda5 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.370702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0800-e329-4b65-946e-9c6c5aeebda5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.371456 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0800-e329-4b65-946e-9c6c5aeebda5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.404197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f487cf-9155-4211-8b02-816da63fd60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.405000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f487cf-9155-4211-8b02-816da63fd60a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.405596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f487cf-9155-4211-8b02-816da63fd60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.405820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f487cf-9155-4211-8b02-816da63fd60a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.406703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f487cf-9155-4211-8b02-816da63fd60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.408688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f487cf-9155-4211-8b02-816da63fd60a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.424991 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 812/1605] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:51.426084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f487cf-9155-4211-8b02-816da63fd60a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.440826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f487cf-9155-4211-8b02-816da63fd60a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.447017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7d066-3981-4ef8-92a3-42770564830f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.448086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7d066-3981-4ef8-92a3-42770564830f admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.448519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7d066-3981-4ef8-92a3-42770564830f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.448904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7d066-3981-4ef8-92a3-42770564830f admin admin] PATH_INFO: `/v3/projects/0e110109075848a89e6ae1d516de9784/users/38ea91bf04e741c793e4a06db9fb046f/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.464323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd0800-e329-4b65-946e-9c6c5aeebda5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.466455 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd0800-e329-4b65-946e-9c6c5aeebda5 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.469894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7d066-3981-4ef8-92a3-42770564830f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e110109075848a89e6ae1d516de9784, user_id=38ea91bf04e741c793e4a06db9fb046f, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.474973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7d066-3981-4ef8-92a3-42770564830f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.478866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 794/1606] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:51.479690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd0800-e329-4b65-946e-9c6c5aeebda5 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.492174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd0800-e329-4b65-946e-9c6c5aeebda5 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.497567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 813/1607] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:51 2022] PUT /identity/v3/projects/0e110109075848a89e6ae1d516de9784/users/38ea91bf04e741c793e4a06db9fb046f/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:51.500034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9b954-ebd4-499e-8005-c7ec6226f1ae None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.500836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9b954-ebd4-499e-8005-c7ec6226f1ae None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.501212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9b954-ebd4-499e-8005-c7ec6226f1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.515134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:51.547646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f3d9b954-ebd4-499e-8005-c7ec6226f1ae None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:51.550038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f3d9b954-ebd4-499e-8005-c7ec6226f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:51.587306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771089a1-6a58-4f01-9e07-a68cff8f7fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.587580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771089a1-6a58-4f01-9e07-a68cff8f7fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.587858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771089a1-6a58-4f01-9e07-a68cff8f7fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.588144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771089a1-6a58-4f01-9e07-a68cff8f7fe2 admin admin] PATH_INFO: `/v3/projects/f5ee5b7f856846e9b6469504ce3f6f6c/users/c5fb1de7101e4a0fae62c82f901e83d3/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.607613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771089a1-6a58-4f01-9e07-a68cff8f7fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ee5b7f856846e9b6469504ce3f6f6c, user_id=c5fb1de7101e4a0fae62c82f901e83d3, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.610615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771089a1-6a58-4f01-9e07-a68cff8f7fe2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.645173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 795/1608] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:51 2022] PUT /identity/v3/projects/f5ee5b7f856846e9b6469504ce3f6f6c/users/c5fb1de7101e4a0fae62c82f901e83d3/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:51.647471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89dd67-fac5-45b9-bbd4-43a00f477337 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.647764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89dd67-fac5-45b9-bbd4-43a00f477337 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.648049 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89dd67-fac5-45b9-bbd4-43a00f477337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.665540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 814/1609] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:51.666509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9b954-ebd4-499e-8005-c7ec6226f1ae None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:51.705379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9b954-ebd4-499e-8005-c7ec6226f1ae None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:51.728566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2c89dd67-fac5-45b9-bbd4-43a00f477337 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:51.733249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2c89dd67-fac5-45b9-bbd4-43a00f477337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:51.861132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54230e83-59c2-462f-b515-30d74620f05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:51.861863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54230e83-59c2-462f-b515-30d74620f05f admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.862152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54230e83-59c2-462f-b515-30d74620f05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.862428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54230e83-59c2-462f-b515-30d74620f05f admin admin] PATH_INFO: `/v3/projects/23acf169856c41d888a1d4c0e61cfee2/users/bcd4602a40964a81931cdadc92d7f281/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.883238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 796/1610] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:51.885101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b0458-22bf-47e6-9a86-38427aee9609 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.885533 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b0458-22bf-47e6-9a86-38427aee9609 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.885904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b0458-22bf-47e6-9a86-38427aee9609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.903238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54230e83-59c2-462f-b515-30d74620f05f admin admin] RBAC: Authorizing `identity:create_grant(project_id=23acf169856c41d888a1d4c0e61cfee2, user_id=bcd4602a40964a81931cdadc92d7f281, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:51.905997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54230e83-59c2-462f-b515-30d74620f05f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:51.918441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-942b0458-22bf-47e6-9a86-38427aee9609 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:51.919971 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-942b0458-22bf-47e6-9a86-38427aee9609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:51.932046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 815/1611] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:51 2022] PUT /identity/v3/projects/23acf169856c41d888a1d4c0e61cfee2/users/bcd4602a40964a81931cdadc92d7f281/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:51.932046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaf486-4f69-430f-af44-2cef32d78921 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:51.932046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaf486-4f69-430f-af44-2cef32d78921 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:51.932046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaf486-4f69-430f-af44-2cef32d78921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:51.944770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:51.990171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-33aaf486-4f69-430f-af44-2cef32d78921 None None] MFA Rules not processed for user `c5fb1de7101e4a0fae62c82f901e83d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:51.992121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-33aaf486-4f69-430f-af44-2cef32d78921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:52.016256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 797/1612] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:52.017648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b0458-22bf-47e6-9a86-38427aee9609 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:52.019372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-942b0458-22bf-47e6-9a86-38427aee9609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:52.117842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b0458-22bf-47e6-9a86-38427aee9609 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:52.120791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 816/1613] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:52.122282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5151e9-3b5f-4091-9e8e-52b957e0691a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.122642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5151e9-3b5f-4091-9e8e-52b957e0691a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.122834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5151e9-3b5f-4091-9e8e-52b957e0691a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.128243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272c825-fd2e-4de5-aaa3-57e9659dd17c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:52.131771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272c825-fd2e-4de5-aaa3-57e9659dd17c service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.131771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272c825-fd2e-4de5-aaa3-57e9659dd17c service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.131771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272c825-fd2e-4de5-aaa3-57e9659dd17c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.158101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272c825-fd2e-4de5-aaa3-57e9659dd17c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:52.158350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6c5151e9-3b5f-4091-9e8e-52b957e0691a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:52.160183 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6c5151e9-3b5f-4091-9e8e-52b957e0691a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:52.163289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272c825-fd2e-4de5-aaa3-57e9659dd17c service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:52.170201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 817/1614] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:52.171063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5151e9-3b5f-4091-9e8e-52b957e0691a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:52.187778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5151e9-3b5f-4091-9e8e-52b957e0691a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:52.193403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 798/1615] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:52.193592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4cf4c-87b0-4b15-8938-2bcc218d08e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:52.194384 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272c825-fd2e-4de5-aaa3-57e9659dd17c service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:52.194521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4cf4c-87b0-4b15-8938-2bcc218d08e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.194667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4cf4c-87b0-4b15-8938-2bcc218d08e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.195066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4cf4c-87b0-4b15-8938-2bcc218d08e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.206010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:52.208249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272c825-fd2e-4de5-aaa3-57e9659dd17c service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:52.209963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4cf4c-87b0-4b15-8938-2bcc218d08e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:52.211977 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4cf4c-87b0-4b15-8938-2bcc218d08e2 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:52.214883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6333eb1-783a-4af6-8b41-29d73c930e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:52.216180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6333eb1-783a-4af6-8b41-29d73c930e39 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.216699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6333eb1-783a-4af6-8b41-29d73c930e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.218003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6333eb1-783a-4af6-8b41-29d73c930e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.227158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 818/1616] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:52.228403 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4cf4c-87b0-4b15-8938-2bcc218d08e2 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:52.236465 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6333eb1-783a-4af6-8b41-29d73c930e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:52.238082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6333eb1-783a-4af6-8b41-29d73c930e39 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:52.241059 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4cf4c-87b0-4b15-8938-2bcc218d08e2 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:52.247445 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f2564-d79e-42ff-b548-e20c5032e579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:52.248389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f2564-d79e-42ff-b548-e20c5032e579 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.249210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f2564-d79e-42ff-b548-e20c5032e579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.249210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f2564-d79e-42ff-b548-e20c5032e579 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.250202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f2564-d79e-42ff-b548-e20c5032e579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:52.252552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f2564-d79e-42ff-b548-e20c5032e579 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:52.252707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 799/1617] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:52.254410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6333eb1-783a-4af6-8b41-29d73c930e39 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:52.266857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6333eb1-783a-4af6-8b41-29d73c930e39 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:52.272323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc6a25-bf1c-4528-84d1-97ef3d494536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:52.273284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc6a25-bf1c-4528-84d1-97ef3d494536 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.273750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc6a25-bf1c-4528-84d1-97ef3d494536 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.274304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc6a25-bf1c-4528-84d1-97ef3d494536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.289221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc6a25-bf1c-4528-84d1-97ef3d494536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:52.290799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc6a25-bf1c-4528-84d1-97ef3d494536 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:52.302317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 819/1618] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:52 2022] POST /identity/v3/projects => generated 363 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:52.304035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 800/1619] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:52.306605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f2564-d79e-42ff-b548-e20c5032e579 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:52.403166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f2564-d79e-42ff-b548-e20c5032e579 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:52.413861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffaf887-3723-40c2-9696-eddcb36c6ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:52.414760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffaf887-3723-40c2-9696-eddcb36c6ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.415609 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffaf887-3723-40c2-9696-eddcb36c6ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.415964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffaf887-3723-40c2-9696-eddcb36c6ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.417179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffaf887-3723-40c2-9696-eddcb36c6ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:52.420389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffaf887-3723-40c2-9696-eddcb36c6ac1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:52.455126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 820/1620] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:52 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:42:52.458762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc6a25-bf1c-4528-84d1-97ef3d494536 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:52.469650 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc6a25-bf1c-4528-84d1-97ef3d494536 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:52.474717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fb3ee-1b9a-485e-b17f-e72e4c4991c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:52.475499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fb3ee-1b9a-485e-b17f-e72e4c4991c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.475781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fb3ee-1b9a-485e-b17f-e72e4c4991c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.476058 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fb3ee-1b9a-485e-b17f-e72e4c4991c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.476736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fb3ee-1b9a-485e-b17f-e72e4c4991c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:52.478357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fb3ee-1b9a-485e-b17f-e72e4c4991c0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:52.493199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 801/1621] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:52.497017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffaf887-3723-40c2-9696-eddcb36c6ac1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:52.510736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffaf887-3723-40c2-9696-eddcb36c6ac1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:52.520583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef33c5-8f44-4eb0-b6c0-37238c23251b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:52.530506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef33c5-8f44-4eb0-b6c0-37238c23251b admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.531084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef33c5-8f44-4eb0-b6c0-37238c23251b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.531554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef33c5-8f44-4eb0-b6c0-37238c23251b admin admin] PATH_INFO: `/v3/projects/a636fe6d814b462c92568abf79709bec/users/4eef2a2f4b7f463fb94a88716f24baa3/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.550215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef33c5-8f44-4eb0-b6c0-37238c23251b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a636fe6d814b462c92568abf79709bec, user_id=4eef2a2f4b7f463fb94a88716f24baa3, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:52.554520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef33c5-8f44-4eb0-b6c0-37238c23251b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:52.572300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 821/1622] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:52 2022] PUT /identity/v3/projects/a636fe6d814b462c92568abf79709bec/users/4eef2a2f4b7f463fb94a88716f24baa3/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:52.576379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fb3ee-1b9a-485e-b17f-e72e4c4991c0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:52.593422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fb3ee-1b9a-485e-b17f-e72e4c4991c0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:52.693064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bea1e-47db-4710-a72f-88a70f1c6b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:52.694056 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bea1e-47db-4710-a72f-88a70f1c6b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.694479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bea1e-47db-4710-a72f-88a70f1c6b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.694791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bea1e-47db-4710-a72f-88a70f1c6b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.695633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bea1e-47db-4710-a72f-88a70f1c6b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:52.698949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bea1e-47db-4710-a72f-88a70f1c6b92 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:52.715336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 802/1623] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:52.720817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef33c5-8f44-4eb0-b6c0-37238c23251b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:52.735094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef33c5-8f44-4eb0-b6c0-37238c23251b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:52.743668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791f5f9-4d43-4f02-9147-862523c0ce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:52.744844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791f5f9-4d43-4f02-9147-862523c0ce12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.745963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791f5f9-4d43-4f02-9147-862523c0ce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.745963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791f5f9-4d43-4f02-9147-862523c0ce12 admin admin] PATH_INFO: `/v3/projects/a636fe6d814b462c92568abf79709bec/users/4eef2a2f4b7f463fb94a88716f24baa3/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.748243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791f5f9-4d43-4f02-9147-862523c0ce12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a636fe6d814b462c92568abf79709bec, user_id=4eef2a2f4b7f463fb94a88716f24baa3, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:52.751660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791f5f9-4d43-4f02-9147-862523c0ce12 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:52.777516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 822/1624] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:52 2022] PUT /identity/v3/projects/a636fe6d814b462c92568abf79709bec/users/4eef2a2f4b7f463fb94a88716f24baa3/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:52.778945 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99271b1f-dbe3-4f3f-9719-3be14efc6b09 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:52.779333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99271b1f-dbe3-4f3f-9719-3be14efc6b09 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:52.780096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99271b1f-dbe3-4f3f-9719-3be14efc6b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:52.833432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-99271b1f-dbe3-4f3f-9719-3be14efc6b09 None None] MFA Rules not processed for user `4eef2a2f4b7f463fb94a88716f24baa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:52.835413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-99271b1f-dbe3-4f3f-9719-3be14efc6b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:52.854242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:53.021188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85f8d0-247c-4abf-8cfc-40a539ad4804 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:53.021748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85f8d0-247c-4abf-8cfc-40a539ad4804 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:53.022046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85f8d0-247c-4abf-8cfc-40a539ad4804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:53.029421 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 803/1625] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:42:53.046809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99271b1f-dbe3-4f3f-9719-3be14efc6b09 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:53.098957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0b85f8d0-247c-4abf-8cfc-40a539ad4804 None None] MFA Rules not processed for user `78ce88f533c04055bcc81f39d31da988`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:53.100208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0b85f8d0-247c-4abf-8cfc-40a539ad4804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:53.157209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99271b1f-dbe3-4f3f-9719-3be14efc6b09 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:53.163806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1706365-a83e-476b-81f8-3506a0ae5ad3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:53.165175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1706365-a83e-476b-81f8-3506a0ae5ad3 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:53.165608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1706365-a83e-476b-81f8-3506a0ae5ad3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:53.166040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1706365-a83e-476b-81f8-3506a0ae5ad3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:53.184694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1706365-a83e-476b-81f8-3506a0ae5ad3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:53.187016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1706365-a83e-476b-81f8-3506a0ae5ad3 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:53.233956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 804/1626] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:42:53.306344 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 823/1627] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:53 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:53.308373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1706365-a83e-476b-81f8-3506a0ae5ad3 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:53.396167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85f8d0-247c-4abf-8cfc-40a539ad4804 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:53.413520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85f8d0-247c-4abf-8cfc-40a539ad4804 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:53.416749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1706365-a83e-476b-81f8-3506a0ae5ad3 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:53.427694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4899978-8d61-4ab7-be69-269c51dcb04a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:53.428350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4899978-8d61-4ab7-be69-269c51dcb04a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:53.428691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4899978-8d61-4ab7-be69-269c51dcb04a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:53.428958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4899978-8d61-4ab7-be69-269c51dcb04a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:53.430273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab5907-3bab-4ded-8e6f-1791781bf081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:53.432702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab5907-3bab-4ded-8e6f-1791781bf081 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:53.433487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab5907-3bab-4ded-8e6f-1791781bf081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:53.434203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab5907-3bab-4ded-8e6f-1791781bf081 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:53.435405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab5907-3bab-4ded-8e6f-1791781bf081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:53.440162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab5907-3bab-4ded-8e6f-1791781bf081 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:53.455516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:53.503568 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 805/1628] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:53 2022] POST /identity/v3/projects => generated 405 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:53.506092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab5907-3bab-4ded-8e6f-1791781bf081 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:53.554025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4899978-8d61-4ab7-be69-269c51dcb04a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:53.555496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4899978-8d61-4ab7-be69-269c51dcb04a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:53.577933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 824/1629] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:42:53.588677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:53.689889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab5907-3bab-4ded-8e6f-1791781bf081 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:53.703690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd9946-87ea-4cdf-bc52-0abe0e01e77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:53.703980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd9946-87ea-4cdf-bc52-0abe0e01e77a admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:53.704277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd9946-87ea-4cdf-bc52-0abe0e01e77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:53.704833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd9946-87ea-4cdf-bc52-0abe0e01e77a admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:53.706016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd9946-87ea-4cdf-bc52-0abe0e01e77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:53.708242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd9946-87ea-4cdf-bc52-0abe0e01e77a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:53.772585 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 806/1630] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:53 2022] POST /identity/v3/users => generated 309 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:42:53.773252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4899978-8d61-4ab7-be69-269c51dcb04a service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:53.798642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4899978-8d61-4ab7-be69-269c51dcb04a service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:53.809168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7186c6-bab6-49b1-9f2a-f5561254df65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:53.809823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7186c6-bab6-49b1-9f2a-f5561254df65 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:53.810112 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7186c6-bab6-49b1-9f2a-f5561254df65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:53.810379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7186c6-bab6-49b1-9f2a-f5561254df65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:53.812121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7186c6-bab6-49b1-9f2a-f5561254df65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:53.812928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7186c6-bab6-49b1-9f2a-f5561254df65 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:53.846071 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 825/1631] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:53.847226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd9946-87ea-4cdf-bc52-0abe0e01e77a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:53.859289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd9946-87ea-4cdf-bc52-0abe0e01e77a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:53.865376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc159a-e529-4b43-9d5c-1c81038666ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:53.866177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc159a-e529-4b43-9d5c-1c81038666ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:53.866530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc159a-e529-4b43-9d5c-1c81038666ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:53.866813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc159a-e529-4b43-9d5c-1c81038666ce admin admin] PATH_INFO: `/v3/projects/a0446ec4324042f59ddb59d867bffd8d/users/3b43094fc7464e58bf2fba38e75308db/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:53.895749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc159a-e529-4b43-9d5c-1c81038666ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0446ec4324042f59ddb59d867bffd8d, user_id=3b43094fc7464e58bf2fba38e75308db, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:53.899820 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc159a-e529-4b43-9d5c-1c81038666ce admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:53.928261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 807/1632] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:53 2022] PUT /identity/v3/projects/a0446ec4324042f59ddb59d867bffd8d/users/3b43094fc7464e58bf2fba38e75308db/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:53.929535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7186c6-bab6-49b1-9f2a-f5561254df65 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:53.953533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7186c6-bab6-49b1-9f2a-f5561254df65 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:54.099691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ca257-c660-4df1-b1d3-17cf482d074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:54.104819 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca257-c660-4df1-b1d3-17cf482d074b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:54.106098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca257-c660-4df1-b1d3-17cf482d074b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:54.106886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca257-c660-4df1-b1d3-17cf482d074b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:54.108397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ca257-c660-4df1-b1d3-17cf482d074b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:54.112883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ca257-c660-4df1-b1d3-17cf482d074b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:54.148540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 826/1633] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:54.149879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc159a-e529-4b43-9d5c-1c81038666ce admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:54.167803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc159a-e529-4b43-9d5c-1c81038666ce admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:54.179161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e90177-cb9c-4eb5-99c9-9f34b8507305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:54.179881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90177-cb9c-4eb5-99c9-9f34b8507305 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:54.180164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90177-cb9c-4eb5-99c9-9f34b8507305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:54.180441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90177-cb9c-4eb5-99c9-9f34b8507305 admin admin] PATH_INFO: `/v3/projects/a0446ec4324042f59ddb59d867bffd8d/users/3b43094fc7464e58bf2fba38e75308db/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:54.187059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e90177-cb9c-4eb5-99c9-9f34b8507305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0446ec4324042f59ddb59d867bffd8d, user_id=3b43094fc7464e58bf2fba38e75308db, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:54.190217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e90177-cb9c-4eb5-99c9-9f34b8507305 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:54.219364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 808/1634] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:54 2022] PUT /identity/v3/projects/a0446ec4324042f59ddb59d867bffd8d/users/3b43094fc7464e58bf2fba38e75308db/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:54.399303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd21d1f-ab1a-4dac-866d-22a9517a4ba0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:54.399597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd21d1f-ab1a-4dac-866d-22a9517a4ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:54.399988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd21d1f-ab1a-4dac-866d-22a9517a4ba0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:54.402008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 827/1635] 10.222.0.60 () {54 vars in 962 bytes} [Wed Aug 31 11:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:42:54.412804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221fe2b-89c8-4f31-ad94-7e7e3f6350d9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:54.416699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221fe2b-89c8-4f31-ad94-7e7e3f6350d9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:54.417501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221fe2b-89c8-4f31-ad94-7e7e3f6350d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:54.482836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d221fe2b-89c8-4f31-ad94-7e7e3f6350d9 None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:54.484711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d221fe2b-89c8-4f31-ad94-7e7e3f6350d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:54.579838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 809/1636] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Aug 31 11:42:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:54.589712 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2e400-ac79-4974-8a1d-ccadf00059c4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:54.590312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2e400-ac79-4974-8a1d-ccadf00059c4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:54.591047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2e400-ac79-4974-8a1d-ccadf00059c4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:54.592341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 828/1637] 10.222.0.60 () {54 vars in 912 bytes} [Wed Aug 31 11:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:42:54.601873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be45b3d-636d-43e1-8829-4fb1100081b5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:54.602463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be45b3d-636d-43e1-8829-4fb1100081b5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:54.602463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be45b3d-636d-43e1-8829-4fb1100081b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:54.630647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8be45b3d-636d-43e1-8829-4fb1100081b5 None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:54.632355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8be45b3d-636d-43e1-8829-4fb1100081b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:54.649161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 810/1638] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:42:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:54.661486 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2e400-ac79-4974-8a1d-ccadf00059c4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:54.770988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2e400-ac79-4974-8a1d-ccadf00059c4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:54.776944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886115ca-7467-454f-9959-3dea1c5c5111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:54.778396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886115ca-7467-454f-9959-3dea1c5c5111 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:54.778790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886115ca-7467-454f-9959-3dea1c5c5111 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:54.779496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886115ca-7467-454f-9959-3dea1c5c5111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:54.792933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886115ca-7467-454f-9959-3dea1c5c5111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:54.795085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886115ca-7467-454f-9959-3dea1c5c5111 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:54.817536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 829/1639] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:56.894030 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473e6a1-a251-4764-99e3-0f797a5342e3 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:56.894694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473e6a1-a251-4764-99e3-0f797a5342e3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:56.894694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473e6a1-a251-4764-99e3-0f797a5342e3 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:56.896587 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 811/1640] 10.222.0.60 () {54 vars in 962 bytes} [Wed Aug 31 11:42:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:42:56.908017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d604d58-6591-4d24-8cd2-a8f518ad0aad None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:56.908932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d604d58-6591-4d24-8cd2-a8f518ad0aad None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:56.909479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d604d58-6591-4d24-8cd2-a8f518ad0aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:56.931650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8d604d58-6591-4d24-8cd2-a8f518ad0aad None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:56.933173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8d604d58-6591-4d24-8cd2-a8f518ad0aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:56.944635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 830/1641] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Aug 31 11:42:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:56.950460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643496f5-1567-4844-a39b-710a9baffc67 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:56.952043 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643496f5-1567-4844-a39b-710a9baffc67 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:56.952510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643496f5-1567-4844-a39b-710a9baffc67 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:56.954626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 812/1642] 10.222.0.60 () {54 vars in 912 bytes} [Wed Aug 31 11:42:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:42:56.961099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac9c16-1cb7-4ae5-9aed-0b28404ab5ff None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:56.961630 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac9c16-1cb7-4ae5-9aed-0b28404ab5ff None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:56.962055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac9c16-1cb7-4ae5-9aed-0b28404ab5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:56.992330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-abac9c16-1cb7-4ae5-9aed-0b28404ab5ff None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:56.993836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-abac9c16-1cb7-4ae5-9aed-0b28404ab5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:57.006289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 831/1643] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:42:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:57.021972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643496f5-1567-4844-a39b-710a9baffc67 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:57.038637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643496f5-1567-4844-a39b-710a9baffc67 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:57.044302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c52ea2-28dd-4bc9-a9b0-b0f2b2b0dc78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:57.046126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c52ea2-28dd-4bc9-a9b0-b0f2b2b0dc78 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:57.046906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c52ea2-28dd-4bc9-a9b0-b0f2b2b0dc78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:57.047489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c52ea2-28dd-4bc9-a9b0-b0f2b2b0dc78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:57.076323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c52ea2-28dd-4bc9-a9b0-b0f2b2b0dc78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:57.078449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c52ea2-28dd-4bc9-a9b0-b0f2b2b0dc78 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:57.112411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 813/1644] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:57.585137 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6016e9-0acf-466c-bb94-bc9d47aecf77 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:57.585237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6016e9-0acf-466c-bb94-bc9d47aecf77 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:57.585471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6016e9-0acf-466c-bb94-bc9d47aecf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:57.647813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5e6016e9-0acf-466c-bb94-bc9d47aecf77 None None] MFA Rules not processed for user `3b43094fc7464e58bf2fba38e75308db`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:57.649267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5e6016e9-0acf-466c-bb94-bc9d47aecf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:57.710718 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c52ea2-28dd-4bc9-a9b0-b0f2b2b0dc78 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:57.711407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:57.860424 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:57.892429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c52ea2-28dd-4bc9-a9b0-b0f2b2b0dc78 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:57.897540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 832/1645] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:57 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:57.905644 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:57.906920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:57.907356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:57.907794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] PATH_INFO: `/v3/users/c5fb1de7101e4a0fae62c82f901e83d3` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:57.911165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5fb1de7101e4a0fae62c82f901e83d3)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:57.914204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:42:57.914204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 31 11:42:57.914204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: Aug 31 11:42:57.922837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:58.026147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5fb1de7101e4a0fae62c82f901e83d3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:58.027522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5fb1de7101e4a0fae62c82f901e83d3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:58.042673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5fb1de7101e4a0fae62c82f901e83d3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:58.052895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5fb1de7101e4a0fae62c82f901e83d3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:58.054015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5fb1de7101e4a0fae62c82f901e83d3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:58.088112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 814/1646] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:42:57 2022] DELETE /identity/v3/users/c5fb1de7101e4a0fae62c82f901e83d3 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:58.092342 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1cf59-77c2-4ccb-87e9-74cb4fc2d806 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:58.093293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1cf59-77c2-4ccb-87e9-74cb4fc2d806 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:58.094360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1cf59-77c2-4ccb-87e9-74cb4fc2d806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:58.134979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4ca1cf59-77c2-4ccb-87e9-74cb4fc2d806 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:58.137079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4ca1cf59-77c2-4ccb-87e9-74cb4fc2d806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:58.231481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 833/1647] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:42:58.298751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:58.300473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:58.449225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccba71c-8026-420e-bc77-7efb1a273d78 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:58.459004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fe6a2-9eca-4f06-b41b-02349ceadad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:58.459662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe6a2-9eca-4f06-b41b-02349ceadad4 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:58.459947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe6a2-9eca-4f06-b41b-02349ceadad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:58.460225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe6a2-9eca-4f06-b41b-02349ceadad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:58.482578 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fe6a2-9eca-4f06-b41b-02349ceadad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:58.485643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fe6a2-9eca-4f06-b41b-02349ceadad4 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:58.512151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 815/1648] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:00.929626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1cf59-77c2-4ccb-87e9-74cb4fc2d806 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:00.932797 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4ca1cf59-77c2-4ccb-87e9-74cb4fc2d806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:00.959014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1cf59-77c2-4ccb-87e9-74cb4fc2d806 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:00.975487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:00.976648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:00.977425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:00.977814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] PATH_INFO: `/v3/projects/f5ee5b7f856846e9b6469504ce3f6f6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:00.984402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ee5b7f856846e9b6469504ce3f6f6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:00.985890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:43:00.985890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 31 11:43:00.985890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: Aug 31 11:43:00.987414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:01.123131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ee5b7f856846e9b6469504ce3f6f6c', 'request_id': 'req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:01.154448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ee5b7f856846e9b6469504ce3f6f6c', 'request_id': 'req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:01.164545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ee5b7f856846e9b6469504ce3f6f6c', 'request_id': 'req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:01.189712 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] The token cache is being invalidate because project f5ee5b7f856846e9b6469504ce3f6f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:01.190117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:01.191683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4dac5125-e4bd-4a40-9ad8-038d4fe1797e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:01.193269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 834/1649] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:00 2022] DELETE /identity/v3/projects/f5ee5b7f856846e9b6469504ce3f6f6c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:01.197875 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fe6a2-9eca-4f06-b41b-02349ceadad4 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:01.232732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fe6a2-9eca-4f06-b41b-02349ceadad4 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:01.357427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10643bbf-55db-4a0c-bc0a-e996033832e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:01.358079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10643bbf-55db-4a0c-bc0a-e996033832e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:01.358991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10643bbf-55db-4a0c-bc0a-e996033832e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:01.359275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10643bbf-55db-4a0c-bc0a-e996033832e4 admin admin] PATH_INFO: `/v3/users/4eef2a2f4b7f463fb94a88716f24baa3` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:01.360719 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10643bbf-55db-4a0c-bc0a-e996033832e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eef2a2f4b7f463fb94a88716f24baa3)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:01.362878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10643bbf-55db-4a0c-bc0a-e996033832e4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:01.390721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:01.454503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-10643bbf-55db-4a0c-bc0a-e996033832e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eef2a2f4b7f463fb94a88716f24baa3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:01.456092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-10643bbf-55db-4a0c-bc0a-e996033832e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eef2a2f4b7f463fb94a88716f24baa3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:01.464777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-10643bbf-55db-4a0c-bc0a-e996033832e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eef2a2f4b7f463fb94a88716f24baa3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:01.474073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-10643bbf-55db-4a0c-bc0a-e996033832e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eef2a2f4b7f463fb94a88716f24baa3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:01.475089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-10643bbf-55db-4a0c-bc0a-e996033832e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eef2a2f4b7f463fb94a88716f24baa3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:01.498385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 816/1650] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:01 2022] DELETE /identity/v3/users/4eef2a2f4b7f463fb94a88716f24baa3 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:01.571981 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814a656-8f76-40eb-9384-abe0d5019669 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:01.572261 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814a656-8f76-40eb-9384-abe0d5019669 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:01.572563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814a656-8f76-40eb-9384-abe0d5019669 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:01.574136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 835/1651] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:43:01.648813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9eff3-ca9a-4117-bf2c-4e773b1a816c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:01.649105 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9eff3-ca9a-4117-bf2c-4e773b1a816c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:01.649386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9eff3-ca9a-4117-bf2c-4e773b1a816c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:01.691808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c9d9eff3-ca9a-4117-bf2c-4e773b1a816c None None] MFA Rules not processed for user `f9114a71a3a247188b263d87b6cb12c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:01.693182 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c9d9eff3-ca9a-4117-bf2c-4e773b1a816c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:01.873472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 817/1652] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:43:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:01.943853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814a656-8f76-40eb-9384-abe0d5019669 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:01.946834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4814a656-8f76-40eb-9384-abe0d5019669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:02.017565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814a656-8f76-40eb-9384-abe0d5019669 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:02.048352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41732f42-ba6f-4fa6-8896-f38d854d7bf3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:02.049338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41732f42-ba6f-4fa6-8896-f38d854d7bf3 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:02.049773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41732f42-ba6f-4fa6-8896-f38d854d7bf3 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:02.050232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41732f42-ba6f-4fa6-8896-f38d854d7bf3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:02.063546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41732f42-ba6f-4fa6-8896-f38d854d7bf3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:02.065874 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41732f42-ba6f-4fa6-8896-f38d854d7bf3 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:02.080124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 836/1653] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:04.422286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9eff3-ca9a-4117-bf2c-4e773b1a816c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:04.423965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c9d9eff3-ca9a-4117-bf2c-4e773b1a816c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:04.479019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41732f42-ba6f-4fa6-8896-f38d854d7bf3 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:04.481021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-41732f42-ba6f-4fa6-8896-f38d854d7bf3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:04.559424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:04.667002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9eff3-ca9a-4117-bf2c-4e773b1a816c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:04.672803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:04.690889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:04.691695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:04.692194 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] PATH_INFO: `/v3/users/78ce88f533c04055bcc81f39d31da988` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:04.693845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ce88f533c04055bcc81f39d31da988)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:04.698046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:04.713650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41732f42-ba6f-4fa6-8896-f38d854d7bf3 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:04.736315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:04.739021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:04.739743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:04.741347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:04.742243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] PATH_INFO: `/v3/projects/a636fe6d814b462c92568abf79709bec` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:04.745603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a636fe6d814b462c92568abf79709bec)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:04.758593 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:04.794493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ce88f533c04055bcc81f39d31da988'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.797088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ce88f533c04055bcc81f39d31da988'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.819205 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ce88f533c04055bcc81f39d31da988'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.839742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ce88f533c04055bcc81f39d31da988'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.840799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ce88f533c04055bcc81f39d31da988'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.891429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 818/1654] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:04 2022] DELETE /identity/v3/users/78ce88f533c04055bcc81f39d31da988 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:04.895373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:04.898598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:04.901240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a636fe6d814b462c92568abf79709bec', 'request_id': 'req-4f495cc4-c380-473a-93d1-9d605a2cc948'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.923360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a636fe6d814b462c92568abf79709bec', 'request_id': 'req-4f495cc4-c380-473a-93d1-9d605a2cc948'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.939869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a636fe6d814b462c92568abf79709bec', 'request_id': 'req-4f495cc4-c380-473a-93d1-9d605a2cc948'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.971881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] The token cache is being invalidate because project a636fe6d814b462c92568abf79709bec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:04.973189 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.974158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f495cc4-c380-473a-93d1-9d605a2cc948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.976566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 837/1655] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:04 2022] DELETE /identity/v3/projects/a636fe6d814b462c92568abf79709bec => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:04.981425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e7aec-b276-4b4d-9bab-664a6a435020 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:04.981861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e7aec-b276-4b4d-9bab-664a6a435020 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:04.983911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e7aec-b276-4b4d-9bab-664a6a435020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:05.041076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1e1e7aec-b276-4b4d-9bab-664a6a435020 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:05.045958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1e1e7aec-b276-4b4d-9bab-664a6a435020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:05.087836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f96cf-4b43-40eb-9323-78a3c81cd494 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:05.102259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c44135-ff44-4408-bd6c-3f3b1a676b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:05.105540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c44135-ff44-4408-bd6c-3f3b1a676b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:05.106918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c44135-ff44-4408-bd6c-3f3b1a676b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:05.107355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c44135-ff44-4408-bd6c-3f3b1a676b38 admin admin] PATH_INFO: `/v3/users/2b9465185a57462f823a40711d8c3742` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:05.111843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c44135-ff44-4408-bd6c-3f3b1a676b38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b9465185a57462f823a40711d8c3742)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:05.116672 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c44135-ff44-4408-bd6c-3f3b1a676b38 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:05.218001 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 838/1656] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:05.228644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb35f4-126d-4fb3-924a-68ec8bf17fc0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:05.228644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb35f4-126d-4fb3-924a-68ec8bf17fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:05.228644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb35f4-126d-4fb3-924a-68ec8bf17fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:05.270030 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-87c44135-ff44-4408-bd6c-3f3b1a676b38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b9465185a57462f823a40711d8c3742'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:05.271177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-87c44135-ff44-4408-bd6c-3f3b1a676b38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b9465185a57462f823a40711d8c3742'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:05.301725 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b9fb35f4-126d-4fb3-924a-68ec8bf17fc0 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:05.303699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b9fb35f4-126d-4fb3-924a-68ec8bf17fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:05.310882 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-87c44135-ff44-4408-bd6c-3f3b1a676b38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b9465185a57462f823a40711d8c3742'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:05.331896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-87c44135-ff44-4408-bd6c-3f3b1a676b38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b9465185a57462f823a40711d8c3742'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:05.332901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-87c44135-ff44-4408-bd6c-3f3b1a676b38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b9465185a57462f823a40711d8c3742'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:05.344550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 839/1657] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:05.354905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb35f4-126d-4fb3-924a-68ec8bf17fc0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:05.381070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb35f4-126d-4fb3-924a-68ec8bf17fc0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:05.383034 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 819/1658] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:04 2022] DELETE /identity/v3/users/2b9465185a57462f823a40711d8c3742 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:05.387498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38760-f52f-48f2-8886-afef43e27570 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:05.388639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38760-f52f-48f2-8886-afef43e27570 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:05.389250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38760-f52f-48f2-8886-afef43e27570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:05.432209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f9e38760-f52f-48f2-8886-afef43e27570 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:05.447341 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f9e38760-f52f-48f2-8886-afef43e27570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:05.497484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:05.511914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:05.543811 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5039729-0441-4c7b-87c3-67f6e922913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:05.545511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039729-0441-4c7b-87c3-67f6e922913f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:05.545791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039729-0441-4c7b-87c3-67f6e922913f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:05.546988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039729-0441-4c7b-87c3-67f6e922913f admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:05.549061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5039729-0441-4c7b-87c3-67f6e922913f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:43:05.549396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5039729-0441-4c7b-87c3-67f6e922913f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:05.553429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5039729-0441-4c7b-87c3-67f6e922913f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:05.586588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 840/1659] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:43:05.591300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5039729-0441-4c7b-87c3-67f6e922913f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:05.593313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f5039729-0441-4c7b-87c3-67f6e922913f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:05.621121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5039729-0441-4c7b-87c3-67f6e922913f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:05.623668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 820/1660] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:05.636996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2ae5a-f86a-451a-9c1b-3ed3c181d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:05.656837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2ae5a-f86a-451a-9c1b-3ed3c181d931 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:05.657522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2ae5a-f86a-451a-9c1b-3ed3c181d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:05.657935 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2ae5a-f86a-451a-9c1b-3ed3c181d931 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:05.658818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2ae5a-f86a-451a-9c1b-3ed3c181d931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:05.662531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2ae5a-f86a-451a-9c1b-3ed3c181d931 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:05.721139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 841/1661] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:05 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:05.726045 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e38760-f52f-48f2-8886-afef43e27570 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:05.757140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e38760-f52f-48f2-8886-afef43e27570 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:05.759666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8411f-b3bd-47d0-80c3-bdfccbdbc657 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:05.759958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8411f-b3bd-47d0-80c3-bdfccbdbc657 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:05.760307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8411f-b3bd-47d0-80c3-bdfccbdbc657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:05.826549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-11a8411f-b3bd-47d0-80c3-bdfccbdbc657 None None] MFA Rules not processed for user `bcd4602a40964a81931cdadc92d7f281`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:05.827816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-11a8411f-b3bd-47d0-80c3-bdfccbdbc657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:05.915358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfe001-22b3-4678-a6a9-96b49ec09603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:05.915992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfe001-22b3-4678-a6a9-96b49ec09603 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:05.916278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfe001-22b3-4678-a6a9-96b49ec09603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:05.916574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfe001-22b3-4678-a6a9-96b49ec09603 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:05.917539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfe001-22b3-4678-a6a9-96b49ec09603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:05.921666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfe001-22b3-4678-a6a9-96b49ec09603 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:05.949033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 842/1662] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:05.949997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8411f-b3bd-47d0-80c3-bdfccbdbc657 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:05.962810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-11a8411f-b3bd-47d0-80c3-bdfccbdbc657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:06.022115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 821/1663] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:05 2022] POST /identity/v3/users => generated 293 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:06.023100 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfe001-22b3-4678-a6a9-96b49ec09603 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:06.039953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d3dfe001-22b3-4678-a6a9-96b49ec09603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:06.071711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfe001-22b3-4678-a6a9-96b49ec09603 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:06.081058 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:06.082547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:06.082832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:06.083104 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] PATH_INFO: `/v3/projects/3201370921ff48d292b0cb81fa97d84f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:06.085478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3201370921ff48d292b0cb81fa97d84f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:06.090400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:06.107263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8411f-b3bd-47d0-80c3-bdfccbdbc657 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:06.115314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f2f61-d1cc-4de8-9aad-89fa7c5f848e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:06.115966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f2f61-d1cc-4de8-9aad-89fa7c5f848e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:06.116255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f2f61-d1cc-4de8-9aad-89fa7c5f848e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:06.116560 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f2f61-d1cc-4de8-9aad-89fa7c5f848e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:06.139337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f2f61-d1cc-4de8-9aad-89fa7c5f848e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:06.149557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f2f61-d1cc-4de8-9aad-89fa7c5f848e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:06.204015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 843/1664] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:06.205091 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f2f61-d1cc-4de8-9aad-89fa7c5f848e service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:06.211342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3201370921ff48d292b0cb81fa97d84f', 'request_id': 'req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:06.217948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3201370921ff48d292b0cb81fa97d84f', 'request_id': 'req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:06.221066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-513f2f61-d1cc-4de8-9aad-89fa7c5f848e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:06.231238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3201370921ff48d292b0cb81fa97d84f', 'request_id': 'req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:06.261400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] The token cache is being invalidate because project 3201370921ff48d292b0cb81fa97d84f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:06.261843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:06.262540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:06.264417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 822/1665] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:06 2022] DELETE /identity/v3/projects/3201370921ff48d292b0cb81fa97d84f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:06.264998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:06.268673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:06.358179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:06.382777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:06.405352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f2f61-d1cc-4de8-9aad-89fa7c5f848e service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:06.453748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bf1a1-ca5c-4c1f-a077-4203eb549ccb admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:06.462281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb85d48-8472-4953-a448-a2140ead2356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:06.463307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb85d48-8472-4953-a448-a2140ead2356 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:06.465705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb85d48-8472-4953-a448-a2140ead2356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:06.466234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb85d48-8472-4953-a448-a2140ead2356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:06.467703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb85d48-8472-4953-a448-a2140ead2356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:06.470292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb85d48-8472-4953-a448-a2140ead2356 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:06.505409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd2ff4-219a-46b8-9dc6-563453e1ea2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:06.506419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2ff4-219a-46b8-9dc6-563453e1ea2e service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:06.506827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2ff4-219a-46b8-9dc6-563453e1ea2e service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:06.507205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2ff4-219a-46b8-9dc6-563453e1ea2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:06.511197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d7fd2ff4-219a-46b8-9dc6-563453e1ea2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:06.523914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 823/1666] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:06.525211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb85d48-8472-4953-a448-a2140ead2356 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:06.527067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6fb85d48-8472-4953-a448-a2140ead2356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:06.561980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb85d48-8472-4953-a448-a2140ead2356 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:06.572284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:06.573468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:06.573920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:06.574325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] PATH_INFO: `/v3/users/3b43094fc7464e58bf2fba38e75308db` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:06.575863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b43094fc7464e58bf2fba38e75308db)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:06.578647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:06.639013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd2ff4-219a-46b8-9dc6-563453e1ea2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:06.640671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd2ff4-219a-46b8-9dc6-563453e1ea2e service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:06.663461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 844/1667] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:06.669938 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbacdf5-8ab8-4399-8c4c-7ec009e2b350 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:06.670233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbacdf5-8ab8-4399-8c4c-7ec009e2b350 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:06.670505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbacdf5-8ab8-4399-8c4c-7ec009e2b350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:06.675948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b43094fc7464e58bf2fba38e75308db'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:06.676678 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b43094fc7464e58bf2fba38e75308db'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:06.698442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b43094fc7464e58bf2fba38e75308db'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:06.711240 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b43094fc7464e58bf2fba38e75308db'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:06.712383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b43094fc7464e58bf2fba38e75308db'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:06.739310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8fbacdf5-8ab8-4399-8c4c-7ec009e2b350 None None] MFA Rules not processed for user `38c4d208994a40fe82310adb18bb3547`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:06.739707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8fbacdf5-8ab8-4399-8c4c-7ec009e2b350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:06.752298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 824/1668] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:06 2022] DELETE /identity/v3/users/3b43094fc7464e58bf2fba38e75308db => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:06.753248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:06.764688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:06.929464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e1862-d70e-42f2-bcc7-2b99b4fc81ca admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:06.940303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b9037-6d2c-4ead-b495-563e6e0a4a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:06.940303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9037-6d2c-4ead-b495-563e6e0a4a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:06.940303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9037-6d2c-4ead-b495-563e6e0a4a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:06.940303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9037-6d2c-4ead-b495-563e6e0a4a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:06.940918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b9037-6d2c-4ead-b495-563e6e0a4a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:06.940918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b9037-6d2c-4ead-b495-563e6e0a4a55 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:06.942222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 845/1669] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:06.951166 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 825/1670] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:06.953694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbacdf5-8ab8-4399-8c4c-7ec009e2b350 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:06.955332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8fbacdf5-8ab8-4399-8c4c-7ec009e2b350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:06.962635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b9037-6d2c-4ead-b495-563e6e0a4a55 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:06.982029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbacdf5-8ab8-4399-8c4c-7ec009e2b350 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:06.986746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b9037-6d2c-4ead-b495-563e6e0a4a55 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:06.997576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597b249-5907-42c7-9ada-a48f5983f2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:07.000466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597b249-5907-42c7-9ada-a48f5983f2d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:07.000887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597b249-5907-42c7-9ada-a48f5983f2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:07.001584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597b249-5907-42c7-9ada-a48f5983f2d8 admin admin] PATH_INFO: `/v3/projects/0ba211099b314194bb6b38c842a9b714/users/a3bb6eacf1ec4f6081cf841bfac04620/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:07.013389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:07.037422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597b249-5907-42c7-9ada-a48f5983f2d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba211099b314194bb6b38c842a9b714, user_id=a3bb6eacf1ec4f6081cf841bfac04620, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:07.048239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597b249-5907-42c7-9ada-a48f5983f2d8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:07.075660 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 826/1671] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:06 2022] PUT /identity/v3/projects/0ba211099b314194bb6b38c842a9b714/users/a3bb6eacf1ec4f6081cf841bfac04620/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:07.090922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11579a26-c963-48cb-be18-9b952adaaecf None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:07.090922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11579a26-c963-48cb-be18-9b952adaaecf None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:07.091316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11579a26-c963-48cb-be18-9b952adaaecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:07.137374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d0379-8618-4f70-bfe1-1bbced75e191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:07.139211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d0379-8618-4f70-bfe1-1bbced75e191 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:07.140679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d0379-8618-4f70-bfe1-1bbced75e191 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:07.141080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d0379-8618-4f70-bfe1-1bbced75e191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:07.151125 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-11579a26-c963-48cb-be18-9b952adaaecf None None] MFA Rules not processed for user `a3bb6eacf1ec4f6081cf841bfac04620`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:07.155115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-11579a26-c963-48cb-be18-9b952adaaecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:07.166473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:07.278290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d0379-8618-4f70-bfe1-1bbced75e191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:07.281185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d0379-8618-4f70-bfe1-1bbced75e191 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:07.296513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 846/1672] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:07.346031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 827/1673] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:07.351133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d0379-8618-4f70-bfe1-1bbced75e191 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:07.367663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d0379-8618-4f70-bfe1-1bbced75e191 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:07.584449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64d12b-0ce3-408a-bb2a-da241b57db21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:07.590297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64d12b-0ce3-408a-bb2a-da241b57db21 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:07.590650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64d12b-0ce3-408a-bb2a-da241b57db21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:07.591031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64d12b-0ce3-408a-bb2a-da241b57db21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:07.591846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64d12b-0ce3-408a-bb2a-da241b57db21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:07.593680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64d12b-0ce3-408a-bb2a-da241b57db21 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:07.689256 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 847/1674] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:07 2022] POST /identity/v3/projects => generated 387 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:07.694496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11579a26-c963-48cb-be18-9b952adaaecf None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:07.712361 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11579a26-c963-48cb-be18-9b952adaaecf None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:07.881525 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6819ca8-40f1-4f70-b46d-b9a075c960fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:07.883216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6819ca8-40f1-4f70-b46d-b9a075c960fb admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:07.883717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6819ca8-40f1-4f70-b46d-b9a075c960fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:07.884431 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6819ca8-40f1-4f70-b46d-b9a075c960fb admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:07.885500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6819ca8-40f1-4f70-b46d-b9a075c960fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:07.887932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6819ca8-40f1-4f70-b46d-b9a075c960fb admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:07.927388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 828/1675] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:07 2022] POST /identity/v3/users => generated 300 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:07.932202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64d12b-0ce3-408a-bb2a-da241b57db21 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:07.950069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64d12b-0ce3-408a-bb2a-da241b57db21 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:07.954646 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6819ca8-40f1-4f70-b46d-b9a075c960fb admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:07.956623 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f6819ca8-40f1-4f70-b46d-b9a075c960fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:07.969072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27533596-d89f-45ae-9d66-ab182d5a7c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:07.971072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27533596-d89f-45ae-9d66-ab182d5a7c98 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:07.971746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27533596-d89f-45ae-9d66-ab182d5a7c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:07.972949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27533596-d89f-45ae-9d66-ab182d5a7c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:07.974634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27533596-d89f-45ae-9d66-ab182d5a7c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:07.978434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27533596-d89f-45ae-9d66-ab182d5a7c98 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:07.982464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6819ca8-40f1-4f70-b46d-b9a075c960fb admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:07.997327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:07.998664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:07.999796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:08.000517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] PATH_INFO: `/v3/projects/a0446ec4324042f59ddb59d867bffd8d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:08.002213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0446ec4324042f59ddb59d867bffd8d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:08.008289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:08.012317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 848/1676] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:08.016036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27533596-d89f-45ae-9d66-ab182d5a7c98 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:08.039515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27533596-d89f-45ae-9d66-ab182d5a7c98 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:08.049179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899378d-12f7-44d4-9f25-c3eb2275a8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:08.049924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899378d-12f7-44d4-9f25-c3eb2275a8ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:08.050268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899378d-12f7-44d4-9f25-c3eb2275a8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:08.051576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899378d-12f7-44d4-9f25-c3eb2275a8ee admin admin] PATH_INFO: `/v3/projects/d0958c3a6fd24376a76b718c870b0c45/users/c5ac113582ca4abbb2308c482bfdfdf0/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:08.078507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899378d-12f7-44d4-9f25-c3eb2275a8ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0958c3a6fd24376a76b718c870b0c45, user_id=c5ac113582ca4abbb2308c482bfdfdf0, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:08.083476 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899378d-12f7-44d4-9f25-c3eb2275a8ee admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:08.114300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 849/1677] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:08 2022] PUT /identity/v3/projects/d0958c3a6fd24376a76b718c870b0c45/users/c5ac113582ca4abbb2308c482bfdfdf0/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:08.115558 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899378d-12f7-44d4-9f25-c3eb2275a8ee admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:08.118338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e899378d-12f7-44d4-9f25-c3eb2275a8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:08.121113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0446ec4324042f59ddb59d867bffd8d', 'request_id': 'req-e8987e2a-b96a-4b1c-ac51-45df64259166'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:08.127213 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:08.134994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0446ec4324042f59ddb59d867bffd8d', 'request_id': 'req-e8987e2a-b96a-4b1c-ac51-45df64259166'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:08.158264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0446ec4324042f59ddb59d867bffd8d', 'request_id': 'req-e8987e2a-b96a-4b1c-ac51-45df64259166'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:08.179860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] The token cache is being invalidate because project a0446ec4324042f59ddb59d867bffd8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:08.180270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:08.180990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:08.184346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 829/1678] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:07 2022] DELETE /identity/v3/projects/a0446ec4324042f59ddb59d867bffd8d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:08.185290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:08.189492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:08.246772 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:08.255821 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899378d-12f7-44d4-9f25-c3eb2275a8ee admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:08.265889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:08.266662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:08.266855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:08.267138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] PATH_INFO: `/v3/projects/283f52c7378b41a79ad6803f87b19a07` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:08.271193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=283f52c7378b41a79ad6803f87b19a07)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:08.273876 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:08.373626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8987e2a-b96a-4b1c-ac51-45df64259166 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:08.384412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb36ee-5a96-408a-b12a-2c54a6dbb7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:08.390922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb36ee-5a96-408a-b12a-2c54a6dbb7bc admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:08.391841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb36ee-5a96-408a-b12a-2c54a6dbb7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:08.392800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb36ee-5a96-408a-b12a-2c54a6dbb7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:08.395260 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb36ee-5a96-408a-b12a-2c54a6dbb7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:08.398742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb36ee-5a96-408a-b12a-2c54a6dbb7bc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:08.438083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 830/1679] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:08.439393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20bc4d-618a-4371-a913-1776b48930a8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:08.439770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20bc4d-618a-4371-a913-1776b48930a8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:08.440018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20bc4d-618a-4371-a913-1776b48930a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:08.442294 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '283f52c7378b41a79ad6803f87b19a07', 'request_id': 'req-dd5cdc26-094e-4a77-a5f4-138198274a5a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:08.458907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '283f52c7378b41a79ad6803f87b19a07', 'request_id': 'req-dd5cdc26-094e-4a77-a5f4-138198274a5a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:08.469216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '283f52c7378b41a79ad6803f87b19a07', 'request_id': 'req-dd5cdc26-094e-4a77-a5f4-138198274a5a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:08.488710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7b20bc4d-618a-4371-a913-1776b48930a8 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:08.493492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7b20bc4d-618a-4371-a913-1776b48930a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:08.508757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] The token cache is being invalidate because project 283f52c7378b41a79ad6803f87b19a07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:08.509145 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:08.512240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:08.517290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 850/1680] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:08 2022] DELETE /identity/v3/projects/283f52c7378b41a79ad6803f87b19a07 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:08.517290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:08.517290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:08.658135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cdc26-094e-4a77-a5f4-138198274a5a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:08.661872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 831/1681] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:08.663198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd4357-19a5-4602-a408-7f2a07527523 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:08.663482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd4357-19a5-4602-a408-7f2a07527523 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:08.663757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd4357-19a5-4602-a408-7f2a07527523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:08.671771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f5660-d5f6-4d16-86cc-81ef03ed6a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:08.672678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f5660-d5f6-4d16-86cc-81ef03ed6a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:08.672963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f5660-d5f6-4d16-86cc-81ef03ed6a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:08.673246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f5660-d5f6-4d16-86cc-81ef03ed6a61 admin admin] PATH_INFO: `/v3/projects/d0958c3a6fd24376a76b718c870b0c45/users/c5ac113582ca4abbb2308c482bfdfdf0/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:08.675419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f5660-d5f6-4d16-86cc-81ef03ed6a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0958c3a6fd24376a76b718c870b0c45, user_id=c5ac113582ca4abbb2308c482bfdfdf0, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:08.679243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f5660-d5f6-4d16-86cc-81ef03ed6a61 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:08.702483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 851/1682] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:08 2022] PUT /identity/v3/projects/d0958c3a6fd24376a76b718c870b0c45/users/c5ac113582ca4abbb2308c482bfdfdf0/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:08.705010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83cfc3-f123-4ffc-8bdc-86afb1f16461 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:08.705188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83cfc3-f123-4ffc-8bdc-86afb1f16461 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:08.705454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83cfc3-f123-4ffc-8bdc-86afb1f16461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:08.740144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e0fd4357-19a5-4602-a408-7f2a07527523 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:08.741414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e0fd4357-19a5-4602-a408-7f2a07527523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:08.779093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-df83cfc3-f123-4ffc-8bdc-86afb1f16461 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:08.786623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-df83cfc3-f123-4ffc-8bdc-86afb1f16461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:08.851337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:08.873213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 832/1683] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:08.875482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5770ab-6fe2-4e8a-ad01-8d1edc7176fb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:08.876309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5770ab-6fe2-4e8a-ad01-8d1edc7176fb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:08.882891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5770ab-6fe2-4e8a-ad01-8d1edc7176fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:08.922904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3a5770ab-6fe2-4e8a-ad01-8d1edc7176fb None None] MFA Rules not processed for user `c5ac113582ca4abbb2308c482bfdfdf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:08.925135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3a5770ab-6fe2-4e8a-ad01-8d1edc7176fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:08.931895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 852/1684] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:08.933544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7bc33-b957-418f-a1a7-b78b346d24c3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:08.934130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7bc33-b957-418f-a1a7-b78b346d24c3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:08.934545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7bc33-b957-418f-a1a7-b78b346d24c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:08.979098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9eb7bc33-b957-418f-a1a7-b78b346d24c3 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:08.979870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9eb7bc33-b957-418f-a1a7-b78b346d24c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:09.000979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 853/1685] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:09.004579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7bc33-b957-418f-a1a7-b78b346d24c3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:09.031778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7bc33-b957-418f-a1a7-b78b346d24c3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:09.042893 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb3aec-5cff-45bd-81ba-f87e022111ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:09.045432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb3aec-5cff-45bd-81ba-f87e022111ad admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:09.045820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb3aec-5cff-45bd-81ba-f87e022111ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:09.046060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb3aec-5cff-45bd-81ba-f87e022111ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:09.048676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb3aec-5cff-45bd-81ba-f87e022111ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:43:09.051571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb3aec-5cff-45bd-81ba-f87e022111ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:09.056825 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb3aec-5cff-45bd-81ba-f87e022111ad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:09.063477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:09.088216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 854/1686] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:43:09.089119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb3aec-5cff-45bd-81ba-f87e022111ad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:09.112606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb3aec-5cff-45bd-81ba-f87e022111ad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:09.133127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357e27d-fb37-42dd-8275-6daa325049e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:09.133763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357e27d-fb37-42dd-8275-6daa325049e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:09.134040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357e27d-fb37-42dd-8275-6daa325049e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:09.134307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357e27d-fb37-42dd-8275-6daa325049e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:09.134956 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357e27d-fb37-42dd-8275-6daa325049e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:43:09.135257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357e27d-fb37-42dd-8275-6daa325049e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:09.136886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357e27d-fb37-42dd-8275-6daa325049e2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:09.140686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 833/1687] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:09.141627 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5770ab-6fe2-4e8a-ad01-8d1edc7176fb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:09.168863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 855/1688] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:43:09.168863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357e27d-fb37-42dd-8275-6daa325049e2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:09.170729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b357e27d-fb37-42dd-8275-6daa325049e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:09.179092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5770ab-6fe2-4e8a-ad01-8d1edc7176fb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:09.194607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca4a58-6d8c-4336-bbe3-1042a57f3851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:09.195484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca4a58-6d8c-4336-bbe3-1042a57f3851 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:09.195914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca4a58-6d8c-4336-bbe3-1042a57f3851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:09.196355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca4a58-6d8c-4336-bbe3-1042a57f3851 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:09.197995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca4a58-6d8c-4336-bbe3-1042a57f3851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:09.203574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca4a58-6d8c-4336-bbe3-1042a57f3851 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:09.255458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 834/1689] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:09 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:09.256598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca4a58-6d8c-4336-bbe3-1042a57f3851 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:09.305816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357e27d-fb37-42dd-8275-6daa325049e2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:09.428706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c168ea-17c1-4e0a-ba56-119111d1597b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:09.430282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c168ea-17c1-4e0a-ba56-119111d1597b service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:09.430904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c168ea-17c1-4e0a-ba56-119111d1597b service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:09.431385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c168ea-17c1-4e0a-ba56-119111d1597b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:09.465147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca4a58-6d8c-4336-bbe3-1042a57f3851 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:09.487383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eba5af-6c33-4541-b824-d915f443b0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:09.488944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba5af-6c33-4541-b824-d915f443b0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:09.489146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba5af-6c33-4541-b824-d915f443b0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:09.489528 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba5af-6c33-4541-b824-d915f443b0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:09.495992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eba5af-6c33-4541-b824-d915f443b0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:09.499237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eba5af-6c33-4541-b824-d915f443b0b4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:09.549972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 835/1690] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:09 2022] POST /identity/v3/projects => generated 387 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:09.552390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eba5af-6c33-4541-b824-d915f443b0b4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:09.589517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c168ea-17c1-4e0a-ba56-119111d1597b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:09.591502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c168ea-17c1-4e0a-ba56-119111d1597b service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:09.624172 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 856/1691] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:09.625914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e89c4b-9e23-4242-a2ea-97be4bbded53 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:09.626206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e89c4b-9e23-4242-a2ea-97be4bbded53 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:09.626481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e89c4b-9e23-4242-a2ea-97be4bbded53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:09.647375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eba5af-6c33-4541-b824-d915f443b0b4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:09.653964 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d59a71-bf82-489a-a7c9-e4a0ee73de95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:09.654602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d59a71-bf82-489a-a7c9-e4a0ee73de95 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:09.654883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d59a71-bf82-489a-a7c9-e4a0ee73de95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:09.655313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d59a71-bf82-489a-a7c9-e4a0ee73de95 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:09.656193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d59a71-bf82-489a-a7c9-e4a0ee73de95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:09.658365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d59a71-bf82-489a-a7c9-e4a0ee73de95 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:09.658757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:09.702203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 836/1692] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:09 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:09.703197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d59a71-bf82-489a-a7c9-e4a0ee73de95 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:09.730251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a2e89c4b-9e23-4242-a2ea-97be4bbded53 None None] MFA Rules not processed for user `5de832888ffd4a69ae88cac6b632304e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:09.731494 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a2e89c4b-9e23-4242-a2ea-97be4bbded53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:09.732355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d59a71-bf82-489a-a7c9-e4a0ee73de95 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:09.737552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3f549-a807-4d28-a2af-33cbfe7eddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:09.738756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3f549-a807-4d28-a2af-33cbfe7eddf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:09.739165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3f549-a807-4d28-a2af-33cbfe7eddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:09.739453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3f549-a807-4d28-a2af-33cbfe7eddf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:09.740310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3f549-a807-4d28-a2af-33cbfe7eddf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:09.742963 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3f549-a807-4d28-a2af-33cbfe7eddf5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:09.774843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 837/1693] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:09 2022] POST /identity/v3/users => generated 294 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:09.775923 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3f549-a807-4d28-a2af-33cbfe7eddf5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:09.793387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3f549-a807-4d28-a2af-33cbfe7eddf5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:09.799716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:09.802220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:09.802949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:09.802949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] PATH_INFO: `/v3/users/a3bb6eacf1ec4f6081cf841bfac04620` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:09.804591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3bb6eacf1ec4f6081cf841bfac04620)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:09.806529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:09.824774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:09.875170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 857/1694] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:09.876289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e89c4b-9e23-4242-a2ea-97be4bbded53 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:09.905231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e89c4b-9e23-4242-a2ea-97be4bbded53 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:09.912929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209aeec-dbf7-40fe-9cee-2b831e2e2ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:09.913983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209aeec-dbf7-40fe-9cee-2b831e2e2ece admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:09.914269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209aeec-dbf7-40fe-9cee-2b831e2e2ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:09.914544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209aeec-dbf7-40fe-9cee-2b831e2e2ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:09.915384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209aeec-dbf7-40fe-9cee-2b831e2e2ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:09.916640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3bb6eacf1ec4f6081cf841bfac04620'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:09.917365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3bb6eacf1ec4f6081cf841bfac04620'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:09.917486 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209aeec-dbf7-40fe-9cee-2b831e2e2ece admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:09.931809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 858/1695] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:09.932773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209aeec-dbf7-40fe-9cee-2b831e2e2ece admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:09.942625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3bb6eacf1ec4f6081cf841bfac04620'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:09.953336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209aeec-dbf7-40fe-9cee-2b831e2e2ece admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:09.962431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d0105-fb6a-432e-90c4-7e935bc33706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:09.963249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d0105-fb6a-432e-90c4-7e935bc33706 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:09.963604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d0105-fb6a-432e-90c4-7e935bc33706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:09.963991 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d0105-fb6a-432e-90c4-7e935bc33706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:09.964936 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d0105-fb6a-432e-90c4-7e935bc33706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:09.967236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3bb6eacf1ec4f6081cf841bfac04620'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:09.967881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3bb6eacf1ec4f6081cf841bfac04620'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:09.968099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d0105-fb6a-432e-90c4-7e935bc33706 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:09.987621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 859/1696] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:09.990541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d0105-fb6a-432e-90c4-7e935bc33706 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:09.991808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-014d0105-fb6a-432e-90c4-7e935bc33706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:10.015732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 838/1697] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:09 2022] DELETE /identity/v3/users/a3bb6eacf1ec4f6081cf841bfac04620 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:10.017240 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:10.020557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:10.157460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d0105-fb6a-432e-90c4-7e935bc33706 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:10.169490 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e1ea5-f53e-4899-aa7b-0309efc56b70 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:10.174778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865b7a-bd36-41b9-8a5e-6130d40b64c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:10.175779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865b7a-bd36-41b9-8a5e-6130d40b64c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:10.176253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865b7a-bd36-41b9-8a5e-6130d40b64c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:10.176638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865b7a-bd36-41b9-8a5e-6130d40b64c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:10.177390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865b7a-bd36-41b9-8a5e-6130d40b64c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:10.179247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865b7a-bd36-41b9-8a5e-6130d40b64c8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:10.203190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 839/1698] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:10.203190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865b7a-bd36-41b9-8a5e-6130d40b64c8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:10.206006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3b865b7a-bd36-41b9-8a5e-6130d40b64c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:10.234720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865b7a-bd36-41b9-8a5e-6130d40b64c8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:10.249725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35dc10-ddbe-4444-9e6f-cbaab44be5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:10.250206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35dc10-ddbe-4444-9e6f-cbaab44be5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:10.251610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35dc10-ddbe-4444-9e6f-cbaab44be5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:10.251909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35dc10-ddbe-4444-9e6f-cbaab44be5a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:10.252855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35dc10-ddbe-4444-9e6f-cbaab44be5a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:10.256248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35dc10-ddbe-4444-9e6f-cbaab44be5a6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:10.275521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 840/1699] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:10.276758 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b904-59d6-4140-a00e-0d9c7c9631ca None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:10.277536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b904-59d6-4140-a00e-0d9c7c9631ca None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:10.277825 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b904-59d6-4140-a00e-0d9c7c9631ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:10.306408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a8132-f233-42d4-85b4-3d8e16c3570e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:10.307058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a8132-f233-42d4-85b4-3d8e16c3570e service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:10.307372 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a8132-f233-42d4-85b4-3d8e16c3570e service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:10.307623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a8132-f233-42d4-85b4-3d8e16c3570e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:10.311363 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-306a8132-f233-42d4-85b4-3d8e16c3570e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:10.321368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:10.330757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e6d6b904-59d6-4140-a00e-0d9c7c9631ca None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:10.333515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e6d6b904-59d6-4140-a00e-0d9c7c9631ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:10.352157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 841/1700] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:10.353268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6b904-59d6-4140-a00e-0d9c7c9631ca None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:10.376747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6b904-59d6-4140-a00e-0d9c7c9631ca None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:10.389302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919080d-03f0-4eed-a7da-06befae29aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:10.397228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919080d-03f0-4eed-a7da-06befae29aee admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:10.398088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919080d-03f0-4eed-a7da-06befae29aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:10.398383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919080d-03f0-4eed-a7da-06befae29aee admin admin] PATH_INFO: `/v3/projects/8392e77cd14f48dfa8a4e65d5035a120/users/41d433ebf9f642aabae47f99a0673faa/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:10.435555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919080d-03f0-4eed-a7da-06befae29aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=8392e77cd14f48dfa8a4e65d5035a120, user_id=41d433ebf9f642aabae47f99a0673faa, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:10.448342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919080d-03f0-4eed-a7da-06befae29aee admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:10.461030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a8132-f233-42d4-85b4-3d8e16c3570e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:10.463769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a8132-f233-42d4-85b4-3d8e16c3570e service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:10.484947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 842/1701] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:10 2022] PUT /identity/v3/projects/8392e77cd14f48dfa8a4e65d5035a120/users/41d433ebf9f642aabae47f99a0673faa/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:10.484947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919080d-03f0-4eed-a7da-06befae29aee admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:10.497405 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 860/1702] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:10.499830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a8132-f233-42d4-85b4-3d8e16c3570e service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:10.502707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-306a8132-f233-42d4-85b4-3d8e16c3570e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:10.594248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:10.633622 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919080d-03f0-4eed-a7da-06befae29aee admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:10.640465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a8132-f233-42d4-85b4-3d8e16c3570e service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:10.652620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311b673-57d5-4a3a-bfaa-63d41bf25da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:10.654072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311b673-57d5-4a3a-bfaa-63d41bf25da8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:10.654360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311b673-57d5-4a3a-bfaa-63d41bf25da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:10.654636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311b673-57d5-4a3a-bfaa-63d41bf25da8 admin admin] PATH_INFO: `/v3/projects/8f73bcaf6b9f4d1a8ec8f92a24fa0e68/users/53d551d81acf43199f33139674059224/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:10.701608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311b673-57d5-4a3a-bfaa-63d41bf25da8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f73bcaf6b9f4d1a8ec8f92a24fa0e68, user_id=53d551d81acf43199f33139674059224, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:10.708508 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311b673-57d5-4a3a-bfaa-63d41bf25da8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:10.750195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 843/1703] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:10 2022] PUT /identity/v3/projects/8f73bcaf6b9f4d1a8ec8f92a24fa0e68/users/53d551d81acf43199f33139674059224/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:10.750195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ea4e-4886-4378-ac56-156c12b95333 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:10.750195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ea4e-4886-4378-ac56-156c12b95333 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:10.750195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ea4e-4886-4378-ac56-156c12b95333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:10.813857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb88d2-24ac-46c4-83f1-1aeb0ac03a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:10.814497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb88d2-24ac-46c4-83f1-1aeb0ac03a7c service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:10.818422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb88d2-24ac-46c4-83f1-1aeb0ac03a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:10.819143 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb88d2-24ac-46c4-83f1-1aeb0ac03a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:10.860066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb88d2-24ac-46c4-83f1-1aeb0ac03a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:10.864058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb88d2-24ac-46c4-83f1-1aeb0ac03a7c service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:10.866590 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3df0ea4e-4886-4378-ac56-156c12b95333 None None] MFA Rules not processed for user `41d433ebf9f642aabae47f99a0673faa`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:10.870374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3df0ea4e-4886-4378-ac56-156c12b95333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:10.888631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 861/1704] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:10.889942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8c5fc-252d-492b-81f6-b59b3b971884 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:10.893649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8c5fc-252d-492b-81f6-b59b3b971884 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:10.894164 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8c5fc-252d-492b-81f6-b59b3b971884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:10.962019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-afd8c5fc-252d-492b-81f6-b59b3b971884 None None] MFA Rules not processed for user `53d551d81acf43199f33139674059224`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:10.963269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-afd8c5fc-252d-492b-81f6-b59b3b971884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:11.054814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 844/1705] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:11.060561 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0ea4e-4886-4378-ac56-156c12b95333 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:11.095915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:11.112076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 862/1706] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:11.119536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8c5fc-252d-492b-81f6-b59b3b971884 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:11.215582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0ea4e-4886-4378-ac56-156c12b95333 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:11.229777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759022fc-3da5-44cc-b68d-c35bcc2508b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:11.230427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759022fc-3da5-44cc-b68d-c35bcc2508b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:11.230596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759022fc-3da5-44cc-b68d-c35bcc2508b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:11.233060 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759022fc-3da5-44cc-b68d-c35bcc2508b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:11.233785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759022fc-3da5-44cc-b68d-c35bcc2508b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:11.240237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759022fc-3da5-44cc-b68d-c35bcc2508b3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:11.282575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8c5fc-252d-492b-81f6-b59b3b971884 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:11.299909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4790e5-72e3-455b-a067-e7f9506ee4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:11.301488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4790e5-72e3-455b-a067-e7f9506ee4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:11.301845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4790e5-72e3-455b-a067-e7f9506ee4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:11.302159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4790e5-72e3-455b-a067-e7f9506ee4d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:11.302834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4790e5-72e3-455b-a067-e7f9506ee4d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:11.304793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4790e5-72e3-455b-a067-e7f9506ee4d7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:11.313670 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 845/1707] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:11 2022] POST /identity/v3/projects => generated 373 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:11.318669 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759022fc-3da5-44cc-b68d-c35bcc2508b3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:11.384927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 863/1708] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:11 2022] POST /identity/v3/projects => generated 387 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:11.388134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4790e5-72e3-455b-a067-e7f9506ee4d7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:11.452037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:11.503213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759022fc-3da5-44cc-b68d-c35bcc2508b3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:11.523889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4790e5-72e3-455b-a067-e7f9506ee4d7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:11.531093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a92e88-5a30-4419-a5ca-48a93c7fb25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:11.532404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a92e88-5a30-4419-a5ca-48a93c7fb25a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:11.532716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a92e88-5a30-4419-a5ca-48a93c7fb25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:11.533146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a92e88-5a30-4419-a5ca-48a93c7fb25a admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:11.533807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a92e88-5a30-4419-a5ca-48a93c7fb25a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:11.537350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a92e88-5a30-4419-a5ca-48a93c7fb25a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:11.570013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 864/1709] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:11 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:11.573652 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a92e88-5a30-4419-a5ca-48a93c7fb25a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:11.588851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a92e88-5a30-4419-a5ca-48a93c7fb25a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:11.594736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec88c0a-dc9d-45f4-a53a-fb5a1067d90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:11.595707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88c0a-dc9d-45f4-a53a-fb5a1067d90e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:11.596313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88c0a-dc9d-45f4-a53a-fb5a1067d90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:11.596754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88c0a-dc9d-45f4-a53a-fb5a1067d90e admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:11.598398 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec88c0a-dc9d-45f4-a53a-fb5a1067d90e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:11.600421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec88c0a-dc9d-45f4-a53a-fb5a1067d90e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:11.619204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 865/1710] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:11.636002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec88c0a-dc9d-45f4-a53a-fb5a1067d90e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:11.664825 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec88c0a-dc9d-45f4-a53a-fb5a1067d90e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:11.681529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2970ed6-d721-4d92-aa4c-5f1bc5822aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:11.682225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2970ed6-d721-4d92-aa4c-5f1bc5822aed admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:11.682513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2970ed6-d721-4d92-aa4c-5f1bc5822aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:11.682794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2970ed6-d721-4d92-aa4c-5f1bc5822aed admin admin] PATH_INFO: `/v3/projects/8fa03860f86f485da2ec8debcb0f04b9/users/8d2a1eb47c444051b926320e45e9b72a/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:11.696865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562466f1-fbda-40df-ae1c-330319088605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:11.698053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562466f1-fbda-40df-ae1c-330319088605 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:11.698974 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562466f1-fbda-40df-ae1c-330319088605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:11.699247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562466f1-fbda-40df-ae1c-330319088605 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:11.699894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562466f1-fbda-40df-ae1c-330319088605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:11.702660 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562466f1-fbda-40df-ae1c-330319088605 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:11.715751 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2970ed6-d721-4d92-aa4c-5f1bc5822aed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa03860f86f485da2ec8debcb0f04b9, user_id=8d2a1eb47c444051b926320e45e9b72a, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:11.718542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2970ed6-d721-4d92-aa4c-5f1bc5822aed admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:11.750163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 866/1711] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:11 2022] PUT /identity/v3/projects/8fa03860f86f485da2ec8debcb0f04b9/users/8d2a1eb47c444051b926320e45e9b72a/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:11.754637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2970ed6-d721-4d92-aa4c-5f1bc5822aed admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:11.781217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 846/1712] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:11 2022] POST /identity/v3/users => generated 293 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:11.783333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562466f1-fbda-40df-ae1c-330319088605 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:11.804860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562466f1-fbda-40df-ae1c-330319088605 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:11.839525 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:11.985631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2970ed6-d721-4d92-aa4c-5f1bc5822aed admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:12.009285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760274cd-30a8-4fbf-8bc5-32d77c3ce957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:12.009937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760274cd-30a8-4fbf-8bc5-32d77c3ce957 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:12.010225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760274cd-30a8-4fbf-8bc5-32d77c3ce957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:12.010505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760274cd-30a8-4fbf-8bc5-32d77c3ce957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:12.011173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760274cd-30a8-4fbf-8bc5-32d77c3ce957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:12.013500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760274cd-30a8-4fbf-8bc5-32d77c3ce957 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:12.014432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ef5c0-71bb-4234-9173-e455aa311dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:12.015302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ef5c0-71bb-4234-9173-e455aa311dec admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:12.015920 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ef5c0-71bb-4234-9173-e455aa311dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:12.016908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ef5c0-71bb-4234-9173-e455aa311dec admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:12.019953 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ef5c0-71bb-4234-9173-e455aa311dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:12.024162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ef5c0-71bb-4234-9173-e455aa311dec admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:12.042499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 847/1713] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:12.046207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760274cd-30a8-4fbf-8bc5-32d77c3ce957 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:12.055511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 867/1714] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:12.060469 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ef5c0-71bb-4234-9173-e455aa311dec admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:12.067126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760274cd-30a8-4fbf-8bc5-32d77c3ce957 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:12.087334 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943f9d7-d281-4a44-8495-d71d695833dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:12.088091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943f9d7-d281-4a44-8495-d71d695833dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:12.088398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943f9d7-d281-4a44-8495-d71d695833dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:12.088730 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943f9d7-d281-4a44-8495-d71d695833dd admin admin] PATH_INFO: `/v3/projects/ea02af9022264c3e83f75edddc9951fa/users/47b06ddebb76462aa75f802e833ba179/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:12.100185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ef5c0-71bb-4234-9173-e455aa311dec admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:12.107161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863754e-6b6c-4d28-a455-b34f89d2c11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:12.109066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863754e-6b6c-4d28-a455-b34f89d2c11e admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:12.109605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863754e-6b6c-4d28-a455-b34f89d2c11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:12.110090 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863754e-6b6c-4d28-a455-b34f89d2c11e admin admin] PATH_INFO: `/v3/projects/8fa03860f86f485da2ec8debcb0f04b9/users/8d2a1eb47c444051b926320e45e9b72a/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:12.113307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863754e-6b6c-4d28-a455-b34f89d2c11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa03860f86f485da2ec8debcb0f04b9, user_id=8d2a1eb47c444051b926320e45e9b72a, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:12.117159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863754e-6b6c-4d28-a455-b34f89d2c11e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:12.150623 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943f9d7-d281-4a44-8495-d71d695833dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea02af9022264c3e83f75edddc9951fa, user_id=47b06ddebb76462aa75f802e833ba179, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:12.153354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943f9d7-d281-4a44-8495-d71d695833dd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:12.166953 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 868/1715] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:12 2022] PUT /identity/v3/projects/8fa03860f86f485da2ec8debcb0f04b9/users/8d2a1eb47c444051b926320e45e9b72a/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:12.180830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68c3d9-7124-4317-9753-a00eff0bef74 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:12.182082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68c3d9-7124-4317-9753-a00eff0bef74 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:12.182593 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68c3d9-7124-4317-9753-a00eff0bef74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:12.202686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 848/1716] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:12 2022] PUT /identity/v3/projects/ea02af9022264c3e83f75edddc9951fa/users/47b06ddebb76462aa75f802e833ba179/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:12.206690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943f9d7-d281-4a44-8495-d71d695833dd admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:12.238636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9a68c3d9-7124-4317-9753-a00eff0bef74 None None] MFA Rules not processed for user `8d2a1eb47c444051b926320e45e9b72a`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:12.240406 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a68c3d9-7124-4317-9753-a00eff0bef74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:12.337403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:12.373462 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943f9d7-d281-4a44-8495-d71d695833dd admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:12.403228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc935fa-0a1f-4ef2-80be-2738734cbee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:12.406094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc935fa-0a1f-4ef2-80be-2738734cbee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:12.406389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc935fa-0a1f-4ef2-80be-2738734cbee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:12.406664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc935fa-0a1f-4ef2-80be-2738734cbee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:12.407359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc935fa-0a1f-4ef2-80be-2738734cbee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:12.409274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc935fa-0a1f-4ef2-80be-2738734cbee6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:12.424317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 869/1717] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:12.440779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68c3d9-7124-4317-9753-a00eff0bef74 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:12.442331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 849/1718] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:43:12.443832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a68c3d9-7124-4317-9753-a00eff0bef74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:12.446367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc935fa-0a1f-4ef2-80be-2738734cbee6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:12.468884 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc935fa-0a1f-4ef2-80be-2738734cbee6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:12.482690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6471c2-3f63-4f76-a210-b6153a15271f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:12.490249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6471c2-3f63-4f76-a210-b6153a15271f admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:12.490882 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6471c2-3f63-4f76-a210-b6153a15271f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:12.492433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6471c2-3f63-4f76-a210-b6153a15271f admin admin] PATH_INFO: `/v3/projects/ea02af9022264c3e83f75edddc9951fa/users/47b06ddebb76462aa75f802e833ba179/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:12.497815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6471c2-3f63-4f76-a210-b6153a15271f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea02af9022264c3e83f75edddc9951fa, user_id=47b06ddebb76462aa75f802e833ba179, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:12.503757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6471c2-3f63-4f76-a210-b6153a15271f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:12.534211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 850/1719] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:12 2022] PUT /identity/v3/projects/ea02af9022264c3e83f75edddc9951fa/users/47b06ddebb76462aa75f802e833ba179/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:12.537810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996ebe8-2ba5-45f9-bd8f-37d4e69113c7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:12.538214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996ebe8-2ba5-45f9-bd8f-37d4e69113c7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:12.539006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996ebe8-2ba5-45f9-bd8f-37d4e69113c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:12.627300 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68c3d9-7124-4317-9753-a00eff0bef74 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:12.633569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2996ebe8-2ba5-45f9-bd8f-37d4e69113c7 None None] MFA Rules not processed for user `47b06ddebb76462aa75f802e833ba179`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:12.641642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2996ebe8-2ba5-45f9-bd8f-37d4e69113c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:12.759685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5b04d-963b-4487-8365-cefd1ce333fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:12.760344 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5b04d-963b-4487-8365-cefd1ce333fa service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:12.760719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5b04d-963b-4487-8365-cefd1ce333fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:12.760997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5b04d-963b-4487-8365-cefd1ce333fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:12.786889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:12.834150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 851/1720] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:12.845529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996ebe8-2ba5-45f9-bd8f-37d4e69113c7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:12.849247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2996ebe8-2ba5-45f9-bd8f-37d4e69113c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:12.873850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996ebe8-2ba5-45f9-bd8f-37d4e69113c7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:12.909017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57943ca2-da1d-421a-91e1-6dd21ef5ebea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:12.911687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57943ca2-da1d-421a-91e1-6dd21ef5ebea service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:12.912374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57943ca2-da1d-421a-91e1-6dd21ef5ebea service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:12.912937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57943ca2-da1d-421a-91e1-6dd21ef5ebea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:12.935660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5b04d-963b-4487-8365-cefd1ce333fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:12.937178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5b04d-963b-4487-8365-cefd1ce333fa service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:12.939829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57943ca2-da1d-421a-91e1-6dd21ef5ebea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:12.943362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57943ca2-da1d-421a-91e1-6dd21ef5ebea service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:12.964559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 870/1721] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:43:12.999495 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 852/1722] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:43:13.103023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5b04d-963b-4487-8365-cefd1ce333fa service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:13.146354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57943ca2-da1d-421a-91e1-6dd21ef5ebea service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:13.157844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57943ca2-da1d-421a-91e1-6dd21ef5ebea service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:13.165258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835eba05-7788-4570-a84e-b362c899465d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:13.165897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eba05-7788-4570-a84e-b362c899465d service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:13.166279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eba05-7788-4570-a84e-b362c899465d service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:13.166558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eba05-7788-4570-a84e-b362c899465d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:13.311332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835eba05-7788-4570-a84e-b362c899465d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:13.311949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5b04d-963b-4487-8365-cefd1ce333fa service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:13.312775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835eba05-7788-4570-a84e-b362c899465d service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:13.329341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c40d4-5536-456c-b52f-6ce0c6d298dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:13.330012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c40d4-5536-456c-b52f-6ce0c6d298dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:13.330301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c40d4-5536-456c-b52f-6ce0c6d298dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:13.330581 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c40d4-5536-456c-b52f-6ce0c6d298dc admin admin] PATH_INFO: `/v3/users/53d551d81acf43199f33139674059224` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:13.333007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c40d4-5536-456c-b52f-6ce0c6d298dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d551d81acf43199f33139674059224)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:13.335010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c40d4-5536-456c-b52f-6ce0c6d298dc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:13.336266 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 853/1723] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:13.436468 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-de6c40d4-5536-456c-b52f-6ce0c6d298dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d551d81acf43199f33139674059224'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:13.455726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-de6c40d4-5536-456c-b52f-6ce0c6d298dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d551d81acf43199f33139674059224'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:13.456747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-de6c40d4-5536-456c-b52f-6ce0c6d298dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d551d81acf43199f33139674059224'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:13.461491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-de6c40d4-5536-456c-b52f-6ce0c6d298dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d551d81acf43199f33139674059224'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:13.484287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-de6c40d4-5536-456c-b52f-6ce0c6d298dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d551d81acf43199f33139674059224'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:13.529707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 871/1724] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:13 2022] DELETE /identity/v3/users/53d551d81acf43199f33139674059224 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:13.532857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2fd61-3fe5-4d82-8ebc-19f7e3d56125 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:13.533210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2fd61-3fe5-4d82-8ebc-19f7e3d56125 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:13.533474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2fd61-3fe5-4d82-8ebc-19f7e3d56125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:13.612055 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f4f2fd61-3fe5-4d82-8ebc-19f7e3d56125 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:13.614092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f4f2fd61-3fe5-4d82-8ebc-19f7e3d56125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:13.773963 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:13.780776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27fadf-c655-4d43-bbec-8162228a66e2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:13.781029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27fadf-c655-4d43-bbec-8162228a66e2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:13.781330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27fadf-c655-4d43-bbec-8162228a66e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:13.853024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2a27fadf-c655-4d43-bbec-8162228a66e2 None None] MFA Rules not processed for user `0d27f92c992749cfa5502babb56cb5a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:13.854279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2a27fadf-c655-4d43-bbec-8162228a66e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:13.856697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 854/1725] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:13.892261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2fd61-3fe5-4d82-8ebc-19f7e3d56125 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:13.893750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f4f2fd61-3fe5-4d82-8ebc-19f7e3d56125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:13.898065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:13.935256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2fd61-3fe5-4d82-8ebc-19f7e3d56125 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:13.952810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:13.953874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:13.954432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:13.954715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] PATH_INFO: `/v3/projects/0ba211099b314194bb6b38c842a9b714` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:13.957135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba211099b314194bb6b38c842a9b714)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:13.960070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:14.045212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 872/1726] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:14.056528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27fadf-c655-4d43-bbec-8162228a66e2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:14.057695 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2a27fadf-c655-4d43-bbec-8162228a66e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:14.098334 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba211099b314194bb6b38c842a9b714', 'request_id': 'req-5bd50870-3018-43f3-99bb-8fc38b12bf50'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:14.116571 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba211099b314194bb6b38c842a9b714', 'request_id': 'req-5bd50870-3018-43f3-99bb-8fc38b12bf50'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:14.128084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba211099b314194bb6b38c842a9b714', 'request_id': 'req-5bd50870-3018-43f3-99bb-8fc38b12bf50'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:14.146259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27fadf-c655-4d43-bbec-8162228a66e2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:14.155318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] The token cache is being invalidate because project 0ba211099b314194bb6b38c842a9b714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:14.156119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:14.156865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bd50870-3018-43f3-99bb-8fc38b12bf50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:14.164332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 855/1727] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:13 2022] DELETE /identity/v3/projects/0ba211099b314194bb6b38c842a9b714 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:14.164332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ddab3-8c3a-4a33-ae24-d2c48d30cc75 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:14.164610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ddab3-8c3a-4a33-ae24-d2c48d30cc75 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:14.165991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ddab3-8c3a-4a33-ae24-d2c48d30cc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:14.211127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-636ddab3-8c3a-4a33-ae24-d2c48d30cc75 None None] MFA Rules not processed for user `0d27f92c992749cfa5502babb56cb5a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:14.214114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-636ddab3-8c3a-4a33-ae24-d2c48d30cc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:14.227636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeca13-2f13-4415-b1af-c52f2cb90122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:14.228921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeca13-2f13-4415-b1af-c52f2cb90122 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:14.229407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeca13-2f13-4415-b1af-c52f2cb90122 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:14.229759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeca13-2f13-4415-b1af-c52f2cb90122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:14.231968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fdaeca13-2f13-4415-b1af-c52f2cb90122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:14.349275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 856/1728] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:14.355530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ddab3-8c3a-4a33-ae24-d2c48d30cc75 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:14.357690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-636ddab3-8c3a-4a33-ae24-d2c48d30cc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:14.381714 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaeca13-2f13-4415-b1af-c52f2cb90122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:14.388138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaeca13-2f13-4415-b1af-c52f2cb90122 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:14.399167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ddab3-8c3a-4a33-ae24-d2c48d30cc75 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:14.410747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019d264-fd94-4b04-ad14-24fbd032654f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:14.411707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019d264-fd94-4b04-ad14-24fbd032654f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:14.411707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019d264-fd94-4b04-ad14-24fbd032654f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:14.411973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019d264-fd94-4b04-ad14-24fbd032654f admin admin] PATH_INFO: `/v3/users/c5ac113582ca4abbb2308c482bfdfdf0` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:14.415589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019d264-fd94-4b04-ad14-24fbd032654f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5ac113582ca4abbb2308c482bfdfdf0)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:14.417596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019d264-fd94-4b04-ad14-24fbd032654f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:14.419016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 873/1729] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:14.419964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeca13-2f13-4415-b1af-c52f2cb90122 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:14.423305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fdaeca13-2f13-4415-b1af-c52f2cb90122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:14.486679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:14.509330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d019d264-fd94-4b04-ad14-24fbd032654f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5ac113582ca4abbb2308c482bfdfdf0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:14.510257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d019d264-fd94-4b04-ad14-24fbd032654f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5ac113582ca4abbb2308c482bfdfdf0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:14.523548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d019d264-fd94-4b04-ad14-24fbd032654f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5ac113582ca4abbb2308c482bfdfdf0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:14.539003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d019d264-fd94-4b04-ad14-24fbd032654f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5ac113582ca4abbb2308c482bfdfdf0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:14.540554 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d019d264-fd94-4b04-ad14-24fbd032654f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5ac113582ca4abbb2308c482bfdfdf0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:14.546745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:14.582351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeca13-2f13-4415-b1af-c52f2cb90122 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:14.588273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 857/1730] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:14 2022] DELETE /identity/v3/users/c5ac113582ca4abbb2308c482bfdfdf0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:14.685993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d532ed5-bab9-4450-9e8e-df41224ed5e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:14.686715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d532ed5-bab9-4450-9e8e-df41224ed5e4 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:14.687000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d532ed5-bab9-4450-9e8e-df41224ed5e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:14.687301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d532ed5-bab9-4450-9e8e-df41224ed5e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:14.689304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6d532ed5-bab9-4450-9e8e-df41224ed5e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:14.790317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20433986-3141-4058-8563-e30e37be7ce8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:14.790548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20433986-3141-4058-8563-e30e37be7ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:14.790797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20433986-3141-4058-8563-e30e37be7ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:14.807293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d532ed5-bab9-4450-9e8e-df41224ed5e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:14.809007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d532ed5-bab9-4450-9e8e-df41224ed5e4 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:14.828429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-20433986-3141-4058-8563-e30e37be7ce8 None None] MFA Rules not processed for user `bcd4602a40964a81931cdadc92d7f281`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:14.829726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-20433986-3141-4058-8563-e30e37be7ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:14.846265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 874/1731] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:14.960434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 858/1732] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:15.012963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa94e06-2eb6-4dee-b30a-ba3c2dccbbab None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:15.013538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa94e06-2eb6-4dee-b30a-ba3c2dccbbab None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:15.013993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa94e06-2eb6-4dee-b30a-ba3c2dccbbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:15.134674 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bfa94e06-2eb6-4dee-b30a-ba3c2dccbbab None None] MFA Rules not processed for user `38ea91bf04e741c793e4a06db9fb046f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:15.137381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bfa94e06-2eb6-4dee-b30a-ba3c2dccbbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:15.230628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:15.304400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 875/1733] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:15.306211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32e96d-4e9d-48c1-ba0d-d73946ed4ee6 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:15.306510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32e96d-4e9d-48c1-ba0d-d73946ed4ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:15.306793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32e96d-4e9d-48c1-ba0d-d73946ed4ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:15.356712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2a32e96d-4e9d-48c1-ba0d-d73946ed4ee6 None None] MFA Rules not processed for user `38ea91bf04e741c793e4a06db9fb046f`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:15.358198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2a32e96d-4e9d-48c1-ba0d-d73946ed4ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:15.380252 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 859/1734] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:15.810198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa94e06-2eb6-4dee-b30a-ba3c2dccbbab None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:15.812562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bfa94e06-2eb6-4dee-b30a-ba3c2dccbbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:16.026107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3202e31-de98-4b38-a026-62c9e4f52334 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:16.028480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa94e06-2eb6-4dee-b30a-ba3c2dccbbab None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:16.028975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3202e31-de98-4b38-a026-62c9e4f52334 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:16.030186 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3202e31-de98-4b38-a026-62c9e4f52334 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:16.041013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 860/1735] 10.222.0.60 () {54 vars in 912 bytes} [Wed Aug 31 11:43:16 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:43:16.043264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3202e31-de98-4b38-a026-62c9e4f52334 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:16.045325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c3202e31-de98-4b38-a026-62c9e4f52334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:16.045583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:16.047970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:16.051364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:16.054277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] PATH_INFO: `/v3/projects/d0958c3a6fd24376a76b718c870b0c45` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:16.057835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0958c3a6fd24376a76b718c870b0c45)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:16.059837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:16.079642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3202e31-de98-4b38-a026-62c9e4f52334 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:16.085653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:16.086593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:16.087587 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:16.088069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] PATH_INFO: `/v3/projects/8f73bcaf6b9f4d1a8ec8f92a24fa0e68` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:16.090395 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f73bcaf6b9f4d1a8ec8f92a24fa0e68)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:16.092771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:16.229153 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0958c3a6fd24376a76b718c870b0c45', 'request_id': 'req-f0c03f6a-af37-4f0c-bc7b-1112dd178991'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.243542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0958c3a6fd24376a76b718c870b0c45', 'request_id': 'req-f0c03f6a-af37-4f0c-bc7b-1112dd178991'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.259711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f73bcaf6b9f4d1a8ec8f92a24fa0e68', 'request_id': 'req-064b6660-50d5-4c61-8782-3f021bf700e2'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.272185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f73bcaf6b9f4d1a8ec8f92a24fa0e68', 'request_id': 'req-064b6660-50d5-4c61-8782-3f021bf700e2'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.274321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0958c3a6fd24376a76b718c870b0c45', 'request_id': 'req-f0c03f6a-af37-4f0c-bc7b-1112dd178991'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.294344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f73bcaf6b9f4d1a8ec8f92a24fa0e68', 'request_id': 'req-064b6660-50d5-4c61-8782-3f021bf700e2'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.303448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] The token cache is being invalidate because project d0958c3a6fd24376a76b718c870b0c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:16.304000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.308325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f0c03f6a-af37-4f0c-bc7b-1112dd178991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.310496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 876/1736] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:15 2022] DELETE /identity/v3/projects/d0958c3a6fd24376a76b718c870b0c45 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:16.311889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e603f-26f9-434b-bec0-f3b1e6820ce1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:16.312308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e603f-26f9-434b-bec0-f3b1e6820ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:16.312782 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e603f-26f9-434b-bec0-f3b1e6820ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:16.320090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] The token cache is being invalidate because project 8f73bcaf6b9f4d1a8ec8f92a24fa0e68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:16.321114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.322974 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-064b6660-50d5-4c61-8782-3f021bf700e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.333669 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 861/1737] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:16 2022] DELETE /identity/v3/projects/8f73bcaf6b9f4d1a8ec8f92a24fa0e68 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:16.334671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26460d-5f4b-416a-9406-9717015dba99 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:16.335068 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26460d-5f4b-416a-9406-9717015dba99 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:16.335445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26460d-5f4b-416a-9406-9717015dba99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:16.400953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1a26460d-5f4b-416a-9406-9717015dba99 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:16.404584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1a26460d-5f4b-416a-9406-9717015dba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:16.416622 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-035e603f-26f9-434b-bec0-f3b1e6820ce1 None None] MFA Rules not processed for user `30546ba7f5614f878a9ba322c7c55ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:16.417857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-035e603f-26f9-434b-bec0-f3b1e6820ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:16.575115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 862/1738] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:16.576123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26460d-5f4b-416a-9406-9717015dba99 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:16.577940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1a26460d-5f4b-416a-9406-9717015dba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:16.605273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:16.618556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26460d-5f4b-416a-9406-9717015dba99 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:16.650238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953fdbf-3ab4-4843-92e4-adfef50531ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:16.651259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953fdbf-3ab4-4843-92e4-adfef50531ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:16.656998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953fdbf-3ab4-4843-92e4-adfef50531ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:16.657514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953fdbf-3ab4-4843-92e4-adfef50531ab admin admin] PATH_INFO: `/v3/users/8d2a1eb47c444051b926320e45e9b72a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:16.660760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953fdbf-3ab4-4843-92e4-adfef50531ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d2a1eb47c444051b926320e45e9b72a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:16.665062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953fdbf-3ab4-4843-92e4-adfef50531ab admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:16.679186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 877/1739] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:43:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:16.681205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb8823-d722-48cd-9805-891a4ce5dc73 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:16.681205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb8823-d722-48cd-9805-891a4ce5dc73 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:16.681205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb8823-d722-48cd-9805-891a4ce5dc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:16.753087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8dcb8823-d722-48cd-9805-891a4ce5dc73 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:16.755149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8dcb8823-d722-48cd-9805-891a4ce5dc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:16.772352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 878/1740] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:16.774178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06230448-a839-494d-b8a4-2a65ebae3d94 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:16.774480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06230448-a839-494d-b8a4-2a65ebae3d94 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:16.774758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06230448-a839-494d-b8a4-2a65ebae3d94 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:16.778478 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 879/1741] 10.222.0.60 () {54 vars in 912 bytes} [Wed Aug 31 11:43:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:43:16.780245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06230448-a839-494d-b8a4-2a65ebae3d94 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:16.821385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0953fdbf-3ab4-4843-92e4-adfef50531ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d2a1eb47c444051b926320e45e9b72a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.823668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0953fdbf-3ab4-4843-92e4-adfef50531ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d2a1eb47c444051b926320e45e9b72a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.828332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06230448-a839-494d-b8a4-2a65ebae3d94 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:16.854879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0953fdbf-3ab4-4843-92e4-adfef50531ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d2a1eb47c444051b926320e45e9b72a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.859026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55968c6f-dbb4-44e3-b6db-eb4718fc797b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:16.859672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55968c6f-dbb4-44e3-b6db-eb4718fc797b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:16.859944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55968c6f-dbb4-44e3-b6db-eb4718fc797b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:16.860222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55968c6f-dbb4-44e3-b6db-eb4718fc797b admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:16.861034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55968c6f-dbb4-44e3-b6db-eb4718fc797b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:43:16.861742 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55968c6f-dbb4-44e3-b6db-eb4718fc797b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:16.867631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55968c6f-dbb4-44e3-b6db-eb4718fc797b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:16.870044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:16.887288 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0953fdbf-3ab4-4843-92e4-adfef50531ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d2a1eb47c444051b926320e45e9b72a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.892350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0953fdbf-3ab4-4843-92e4-adfef50531ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d2a1eb47c444051b926320e45e9b72a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:16.925948 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 880/1742] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:43:16.926940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55968c6f-dbb4-44e3-b6db-eb4718fc797b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:16.930900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-55968c6f-dbb4-44e3-b6db-eb4718fc797b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:16.971417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55968c6f-dbb4-44e3-b6db-eb4718fc797b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:16.983057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc61356-d35c-42a2-91db-28ff507ce67c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:16.983701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc61356-d35c-42a2-91db-28ff507ce67c service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:16.983983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc61356-d35c-42a2-91db-28ff507ce67c service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:16.984253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc61356-d35c-42a2-91db-28ff507ce67c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:16.989022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0dc61356-d35c-42a2-91db-28ff507ce67c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:17.002722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 863/1743] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:16 2022] DELETE /identity/v3/users/8d2a1eb47c444051b926320e45e9b72a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:17.011847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66884c88-ac2c-4cc2-876a-c0407b95eed9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:17.012551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66884c88-ac2c-4cc2-876a-c0407b95eed9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:17.012846 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66884c88-ac2c-4cc2-876a-c0407b95eed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:17.021347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc61356-d35c-42a2-91db-28ff507ce67c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:17.025182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc61356-d35c-42a2-91db-28ff507ce67c service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:17.059739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 881/1744] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:17.077202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cb4cc-6c51-48c6-963b-f3f5cb632cfc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:17.077202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cb4cc-6c51-48c6-963b-f3f5cb632cfc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:17.077202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cb4cc-6c51-48c6-963b-f3f5cb632cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:17.104957 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-66884c88-ac2c-4cc2-876a-c0407b95eed9 None None] MFA Rules not processed for user `30546ba7f5614f878a9ba322c7c55ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:17.112171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-66884c88-ac2c-4cc2-876a-c0407b95eed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:17.158586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9e6cb4cc-6c51-48c6-963b-f3f5cb632cfc None None] MFA Rules not processed for user `38c4d208994a40fe82310adb18bb3547`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:17.165594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9e6cb4cc-6c51-48c6-963b-f3f5cb632cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:17.332452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 864/1745] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:43:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:17.335808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66884c88-ac2c-4cc2-876a-c0407b95eed9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:17.340385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-66884c88-ac2c-4cc2-876a-c0407b95eed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:17.420831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 882/1746] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:17.479707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cb4cc-6c51-48c6-963b-f3f5cb632cfc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:17.481195 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9e6cb4cc-6c51-48c6-963b-f3f5cb632cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:17.517491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cb4cc-6c51-48c6-963b-f3f5cb632cfc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:17.539160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a7a7c-91e8-475f-b2e1-12859d768ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:17.539839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a7a7c-91e8-475f-b2e1-12859d768ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:17.540132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a7a7c-91e8-475f-b2e1-12859d768ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:17.540413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a7a7c-91e8-475f-b2e1-12859d768ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:17.561626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a7a7c-91e8-475f-b2e1-12859d768ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:17.563460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a7a7c-91e8-475f-b2e1-12859d768ee4 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:17.588553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 883/1747] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:17.616208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66884c88-ac2c-4cc2-876a-c0407b95eed9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:17.629203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40914e-c4ba-416a-b76a-bc31e12ba1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:17.630173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40914e-c4ba-416a-b76a-bc31e12ba1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:17.630470 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40914e-c4ba-416a-b76a-bc31e12ba1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:17.630773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40914e-c4ba-416a-b76a-bc31e12ba1f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:17.632422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40914e-c4ba-416a-b76a-bc31e12ba1f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:17.640408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40914e-c4ba-416a-b76a-bc31e12ba1f4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:17.719707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 865/1748] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:17 2022] POST /identity/v3/projects => generated 381 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:17.725417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a7a7c-91e8-475f-b2e1-12859d768ee4 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:17.741570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a7a7c-91e8-475f-b2e1-12859d768ee4 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:17.888647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1cbd7e-95f8-441d-a786-34141fc72f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:17.889276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cbd7e-95f8-441d-a786-34141fc72f79 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:17.889555 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cbd7e-95f8-441d-a786-34141fc72f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:17.889831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cbd7e-95f8-441d-a786-34141fc72f79 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:17.890613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1cbd7e-95f8-441d-a786-34141fc72f79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:17.900902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1cbd7e-95f8-441d-a786-34141fc72f79 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:17.944540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 884/1749] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:17 2022] POST /identity/v3/users => generated 291 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:17.945501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40914e-c4ba-416a-b76a-bc31e12ba1f4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:17.956173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40914e-c4ba-416a-b76a-bc31e12ba1f4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:17.982370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb62eab-669b-4370-8c3c-5115d97071c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:17.983151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb62eab-669b-4370-8c3c-5115d97071c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:17.983441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb62eab-669b-4370-8c3c-5115d97071c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:17.983720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb62eab-669b-4370-8c3c-5115d97071c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:17.984380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb62eab-669b-4370-8c3c-5115d97071c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:17.986038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb62eab-669b-4370-8c3c-5115d97071c2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:18.001515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:18.030641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 866/1750] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:18.035491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1cbd7e-95f8-441d-a786-34141fc72f79 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:18.049773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1cbd7e-95f8-441d-a786-34141fc72f79 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:18.063915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c93798-7ba0-4754-bd62-64c2fa984aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:18.064786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c93798-7ba0-4754-bd62-64c2fa984aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:18.065172 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c93798-7ba0-4754-bd62-64c2fa984aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:18.067984 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c93798-7ba0-4754-bd62-64c2fa984aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:18.068868 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c93798-7ba0-4754-bd62-64c2fa984aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:18.072377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c93798-7ba0-4754-bd62-64c2fa984aaa admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:18.095867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 885/1751] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:18.101698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb62eab-669b-4370-8c3c-5115d97071c2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:18.124634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb62eab-669b-4370-8c3c-5115d97071c2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:18.131641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc73ab1-229a-4428-a68f-6ce1b56e715d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:18.132925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc73ab1-229a-4428-a68f-6ce1b56e715d admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:18.133540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc73ab1-229a-4428-a68f-6ce1b56e715d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:18.133827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc73ab1-229a-4428-a68f-6ce1b56e715d admin admin] PATH_INFO: `/v3/projects/5e596987eeee4dce8eb19823b33954d4/users/10d1b9314e994cd3add36f51051caa12/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:18.141458 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c93798-7ba0-4754-bd62-64c2fa984aaa admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:18.143636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-75c93798-7ba0-4754-bd62-64c2fa984aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:18.176187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc73ab1-229a-4428-a68f-6ce1b56e715d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e596987eeee4dce8eb19823b33954d4, user_id=10d1b9314e994cd3add36f51051caa12, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:18.180969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc73ab1-229a-4428-a68f-6ce1b56e715d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:18.212227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 867/1752] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:18 2022] PUT /identity/v3/projects/5e596987eeee4dce8eb19823b33954d4/users/10d1b9314e994cd3add36f51051caa12/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:18.225830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9703f63-db57-4ddb-b841-d41558a3ec92 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:18.226171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9703f63-db57-4ddb-b841-d41558a3ec92 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:18.226386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9703f63-db57-4ddb-b841-d41558a3ec92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:18.327669 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a9703f63-db57-4ddb-b841-d41558a3ec92 None None] MFA Rules not processed for user `10d1b9314e994cd3add36f51051caa12`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:18.331495 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a9703f63-db57-4ddb-b841-d41558a3ec92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:18.332968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:18.397404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c93798-7ba0-4754-bd62-64c2fa984aaa admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:18.458184 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 868/1753] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:18.459294 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9703f63-db57-4ddb-b841-d41558a3ec92 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:18.461061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a9703f63-db57-4ddb-b841-d41558a3ec92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:18.537506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49c84c-6ed9-46ac-a22d-cefebc2348bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:18.538175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c84c-6ed9-46ac-a22d-cefebc2348bd service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:18.538427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c84c-6ed9-46ac-a22d-cefebc2348bd service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:18.539121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c84c-6ed9-46ac-a22d-cefebc2348bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:18.541051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fc49c84c-6ed9-46ac-a22d-cefebc2348bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:18.600383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9703f63-db57-4ddb-b841-d41558a3ec92 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:18.617894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59d654-d554-47d1-aa65-4c6c5de7a0f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:18.621010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59d654-d554-47d1-aa65-4c6c5de7a0f3 service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:18.621540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59d654-d554-47d1-aa65-4c6c5de7a0f3 service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:18.623524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59d654-d554-47d1-aa65-4c6c5de7a0f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:18.630615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6a59d654-d554-47d1-aa65-4c6c5de7a0f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:18.654235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59d654-d554-47d1-aa65-4c6c5de7a0f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:18.657852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59d654-d554-47d1-aa65-4c6c5de7a0f3 service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:18.672335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 869/1754] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:18.673454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59d654-d554-47d1-aa65-4c6c5de7a0f3 service placement] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:18.681375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49c84c-6ed9-46ac-a22d-cefebc2348bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:18.682998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49c84c-6ed9-46ac-a22d-cefebc2348bd service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:18.705734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 886/1755] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:18.709659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c7eef-3e6b-4348-bfa8-e632072a51ce None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:18.711683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c7eef-3e6b-4348-bfa8-e632072a51ce None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:18.712664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c7eef-3e6b-4348-bfa8-e632072a51ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:18.749244 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9c0c7eef-3e6b-4348-bfa8-e632072a51ce None None] MFA Rules not processed for user `5de832888ffd4a69ae88cac6b632304e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:18.751066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c0c7eef-3e6b-4348-bfa8-e632072a51ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:18.893126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:18.905456 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59d654-d554-47d1-aa65-4c6c5de7a0f3 service placement] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:18.916015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 887/1756] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:18.917505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf55d-325d-4262-b595-b7fb6919baa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:18.918049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c7eef-3e6b-4348-bfa8-e632072a51ce None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:18.918628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf55d-325d-4262-b595-b7fb6919baa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:18.919078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf55d-325d-4262-b595-b7fb6919baa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:18.920039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf55d-325d-4262-b595-b7fb6919baa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:18.920736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbf55d-325d-4262-b595-b7fb6919baa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:18.923004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbf55d-325d-4262-b595-b7fb6919baa2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:18.925313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c0c7eef-3e6b-4348-bfa8-e632072a51ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:18.972928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c7eef-3e6b-4348-bfa8-e632072a51ce None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:18.989620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 870/1757] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:18 2022] POST /identity/v3/projects => generated 383 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:18.990792 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf55d-325d-4262-b595-b7fb6919baa2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:19.071179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:19.125023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:19.125823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:19.126199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:19.126433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] PATH_INFO: `/v3/projects/8fa03860f86f485da2ec8debcb0f04b9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:19.127864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa03860f86f485da2ec8debcb0f04b9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:19.130723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:19.143841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf55d-325d-4262-b595-b7fb6919baa2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:19.156781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542bbfb-6bf3-4d00-bb2a-a50a97faa5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:19.160430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542bbfb-6bf3-4d00-bb2a-a50a97faa5fc admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:19.160875 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542bbfb-6bf3-4d00-bb2a-a50a97faa5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:19.161255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542bbfb-6bf3-4d00-bb2a-a50a97faa5fc admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:19.162333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542bbfb-6bf3-4d00-bb2a-a50a97faa5fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:19.164413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542bbfb-6bf3-4d00-bb2a-a50a97faa5fc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:19.220474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 871/1758] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:18 2022] POST /identity/v3/users => generated 298 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:19.225583 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542bbfb-6bf3-4d00-bb2a-a50a97faa5fc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:19.227326 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa03860f86f485da2ec8debcb0f04b9', 'request_id': 'req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:19.239887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa03860f86f485da2ec8debcb0f04b9', 'request_id': 'req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:19.250725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542bbfb-6bf3-4d00-bb2a-a50a97faa5fc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:19.252058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa03860f86f485da2ec8debcb0f04b9', 'request_id': 'req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:19.281020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] The token cache is being invalidate because project 8fa03860f86f485da2ec8debcb0f04b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:19.281501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:19.283739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8653bb9b-04e0-4062-b1ca-c4c4fe62dc95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:19.289316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 888/1759] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:18 2022] DELETE /identity/v3/projects/8fa03860f86f485da2ec8debcb0f04b9 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:19.294086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69250be-22c2-40be-a73a-924f6be721c7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:19.294275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69250be-22c2-40be-a73a-924f6be721c7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:19.294493 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69250be-22c2-40be-a73a-924f6be721c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:19.360716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e69250be-22c2-40be-a73a-924f6be721c7 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:19.361994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e69250be-22c2-40be-a73a-924f6be721c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:19.516529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aa213-9902-4980-a862-7bf81aa5ff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:19.518465 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aa213-9902-4980-a862-7bf81aa5ff4e admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:19.523247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aa213-9902-4980-a862-7bf81aa5ff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:19.523496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aa213-9902-4980-a862-7bf81aa5ff4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:19.528745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213aa213-9902-4980-a862-7bf81aa5ff4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:19.536463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213aa213-9902-4980-a862-7bf81aa5ff4e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:19.590129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 872/1760] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:19.591344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aa213-9902-4980-a862-7bf81aa5ff4e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:19.593112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-213aa213-9902-4980-a862-7bf81aa5ff4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:19.606995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 889/1761] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:19.608578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a35b94-a4db-47d4-8362-5ec1941496e1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:19.608973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a35b94-a4db-47d4-8362-5ec1941496e1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:19.609137 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a35b94-a4db-47d4-8362-5ec1941496e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:19.623708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aa213-9902-4980-a862-7bf81aa5ff4e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:19.635551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51f5d1-fa3d-4ae0-aa65-2277137de986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:19.636242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51f5d1-fa3d-4ae0-aa65-2277137de986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:19.636560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51f5d1-fa3d-4ae0-aa65-2277137de986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:19.636840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51f5d1-fa3d-4ae0-aa65-2277137de986 admin admin] PATH_INFO: `/v3/projects/359313eab1a647d2946ff77a7c3e4632/users/938b4eff73d1434cb0ffa591efee9d88/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:19.656520 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51f5d1-fa3d-4ae0-aa65-2277137de986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=359313eab1a647d2946ff77a7c3e4632, user_id=938b4eff73d1434cb0ffa591efee9d88, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:19.659362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51f5d1-fa3d-4ae0-aa65-2277137de986 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:19.661180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c8a35b94-a4db-47d4-8362-5ec1941496e1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:19.664595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c8a35b94-a4db-47d4-8362-5ec1941496e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:19.688355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 873/1762] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:19 2022] PUT /identity/v3/projects/359313eab1a647d2946ff77a7c3e4632/users/938b4eff73d1434cb0ffa591efee9d88/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:19.692640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51f5d1-fa3d-4ae0-aa65-2277137de986 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:19.693405 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 890/1763] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:19.697699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a35b94-a4db-47d4-8362-5ec1941496e1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:19.720452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a35b94-a4db-47d4-8362-5ec1941496e1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:19.829641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:19.876332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51f5d1-fa3d-4ae0-aa65-2277137de986 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:19.893357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc852b6-aa9f-49c5-9ede-cc6e6c394cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:19.895987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc852b6-aa9f-49c5-9ede-cc6e6c394cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:19.896341 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc852b6-aa9f-49c5-9ede-cc6e6c394cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:19.897267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc852b6-aa9f-49c5-9ede-cc6e6c394cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:19.897540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc852b6-aa9f-49c5-9ede-cc6e6c394cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:19.899281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc852b6-aa9f-49c5-9ede-cc6e6c394cd5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:19.916348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:19.928012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 874/1764] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:19.929328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc852b6-aa9f-49c5-9ede-cc6e6c394cd5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:19.946007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27deac3-bb63-48c4-ac6b-d6c61eefd7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:19.946626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27deac3-bb63-48c4-ac6b-d6c61eefd7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:19.946907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27deac3-bb63-48c4-ac6b-d6c61eefd7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:19.947178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27deac3-bb63-48c4-ac6b-d6c61eefd7e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:19.947909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27deac3-bb63-48c4-ac6b-d6c61eefd7e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:43:19.951557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27deac3-bb63-48c4-ac6b-d6c61eefd7e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:19.958205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27deac3-bb63-48c4-ac6b-d6c61eefd7e7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:19.967216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc852b6-aa9f-49c5-9ede-cc6e6c394cd5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:19.983380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca5f96-bef6-49c9-a0e9-d2be634cf4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:19.984663 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca5f96-bef6-49c9-a0e9-d2be634cf4d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:19.985071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca5f96-bef6-49c9-a0e9-d2be634cf4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:19.985365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca5f96-bef6-49c9-a0e9-d2be634cf4d2 admin admin] PATH_INFO: `/v3/projects/359313eab1a647d2946ff77a7c3e4632/users/938b4eff73d1434cb0ffa591efee9d88/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:19.990137 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 891/1765] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:43:19.990272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca5f96-bef6-49c9-a0e9-d2be634cf4d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=359313eab1a647d2946ff77a7c3e4632, user_id=938b4eff73d1434cb0ffa591efee9d88, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:19.991171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27deac3-bb63-48c4-ac6b-d6c61eefd7e7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:19.991319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca5f96-bef6-49c9-a0e9-d2be634cf4d2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:20.007681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 875/1766] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:19 2022] PUT /identity/v3/projects/359313eab1a647d2946ff77a7c3e4632/users/938b4eff73d1434cb0ffa591efee9d88/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:20.008349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27deac3-bb63-48c4-ac6b-d6c61eefd7e7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:20.029673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f21561-e9f4-4eda-962c-0575bc86be3f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:20.030154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f21561-e9f4-4eda-962c-0575bc86be3f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:20.033183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f21561-e9f4-4eda-962c-0575bc86be3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:20.073948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-55f21561-e9f4-4eda-962c-0575bc86be3f None None] MFA Rules not processed for user `938b4eff73d1434cb0ffa591efee9d88`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:20.076603 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-55f21561-e9f4-4eda-962c-0575bc86be3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:20.238169 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 876/1767] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:20.255207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f21561-e9f4-4eda-962c-0575bc86be3f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:20.257458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-55f21561-e9f4-4eda-962c-0575bc86be3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:20.284513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf828c59-5c54-484a-b856-15c273efc62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:20.285309 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf828c59-5c54-484a-b856-15c273efc62b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:20.285837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf828c59-5c54-484a-b856-15c273efc62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:20.286246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf828c59-5c54-484a-b856-15c273efc62b admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:20.287021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf828c59-5c54-484a-b856-15c273efc62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:20.296570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf828c59-5c54-484a-b856-15c273efc62b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:20.367254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 892/1768] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:19 2022] POST /identity/v3/projects => generated 379 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:43:20.375748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f21561-e9f4-4eda-962c-0575bc86be3f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:20.385796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf828c59-5c54-484a-b856-15c273efc62b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:20.481307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb6b6a-5031-433b-9e65-185d81753c93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:20.482341 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb6b6a-5031-433b-9e65-185d81753c93 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:20.482741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb6b6a-5031-433b-9e65-185d81753c93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:20.483207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb6b6a-5031-433b-9e65-185d81753c93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:20.544121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:20.619401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf828c59-5c54-484a-b856-15c273efc62b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:20.642917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dd358-80e4-4f7a-a261-fe6eb1adb42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:20.643710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dd358-80e4-4f7a-a261-fe6eb1adb42d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:20.644151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dd358-80e4-4f7a-a261-fe6eb1adb42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:20.644606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dd358-80e4-4f7a-a261-fe6eb1adb42d admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:20.645355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dd358-80e4-4f7a-a261-fe6eb1adb42d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:20.653480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dd358-80e4-4f7a-a261-fe6eb1adb42d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:20.713010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb6b6a-5031-433b-9e65-185d81753c93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:20.717808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb6b6a-5031-433b-9e65-185d81753c93 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:20.748447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 893/1769] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:20 2022] POST /identity/v3/users => generated 290 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:20.756319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 877/1770] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:43:20.775374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dd358-80e4-4f7a-a261-fe6eb1adb42d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:20.819020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dd358-80e4-4f7a-a261-fe6eb1adb42d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:20.834473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32b46-5df2-44d1-998f-58bf5f0595cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:20.838090 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32b46-5df2-44d1-998f-58bf5f0595cb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:20.838435 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32b46-5df2-44d1-998f-58bf5f0595cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:20.838730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32b46-5df2-44d1-998f-58bf5f0595cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:20.840787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f32b46-5df2-44d1-998f-58bf5f0595cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:20.845679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f32b46-5df2-44d1-998f-58bf5f0595cb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:20.894822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 894/1771] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:20.896063 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb6b6a-5031-433b-9e65-185d81753c93 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:20.913124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb6b6a-5031-433b-9e65-185d81753c93 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:20.922666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8b820-a73a-4e12-b89b-bcf215205ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:20.924473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8b820-a73a-4e12-b89b-bcf215205ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:20.924794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8b820-a73a-4e12-b89b-bcf215205ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:20.931961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8b820-a73a-4e12-b89b-bcf215205ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:20.931961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8b820-a73a-4e12-b89b-bcf215205ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:20.933745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8b820-a73a-4e12-b89b-bcf215205ef1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:20.997229 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32b46-5df2-44d1-998f-58bf5f0595cb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:20.997485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 878/1772] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:21.028926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32b46-5df2-44d1-998f-58bf5f0595cb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:21.042418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8b820-a73a-4e12-b89b-bcf215205ef1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:21.059655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850f649-e7c9-4172-98a1-0c5e07516fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:21.062049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850f649-e7c9-4172-98a1-0c5e07516fec admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:21.063895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850f649-e7c9-4172-98a1-0c5e07516fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:21.064791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850f649-e7c9-4172-98a1-0c5e07516fec admin admin] PATH_INFO: `/v3/projects/aad73309590d4d58befa7d7258839dda/users/57eef6228a1c4b7eaeb142dc4a74e4b1/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:21.070894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8b820-a73a-4e12-b89b-bcf215205ef1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:21.079300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3401c-23af-4860-9b8a-15042485f549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:21.082409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3401c-23af-4860-9b8a-15042485f549 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:21.082712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3401c-23af-4860-9b8a-15042485f549 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:21.083004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3401c-23af-4860-9b8a-15042485f549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:21.094320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-73e3401c-23af-4860-9b8a-15042485f549 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:21.126188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850f649-e7c9-4172-98a1-0c5e07516fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad73309590d4d58befa7d7258839dda, user_id=57eef6228a1c4b7eaeb142dc4a74e4b1, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:21.140273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850f649-e7c9-4172-98a1-0c5e07516fec admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:21.205910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 895/1773] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:20 2022] PUT /identity/v3/projects/aad73309590d4d58befa7d7258839dda/users/57eef6228a1c4b7eaeb142dc4a74e4b1/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:21.209161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468f02a-c75c-4b09-b1d3-1242dd415064 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:21.209536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468f02a-c75c-4b09-b1d3-1242dd415064 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:21.209887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468f02a-c75c-4b09-b1d3-1242dd415064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:21.306588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3401c-23af-4860-9b8a-15042485f549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:21.314191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3401c-23af-4860-9b8a-15042485f549 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:21.326108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c468f02a-c75c-4b09-b1d3-1242dd415064 None None] MFA Rules not processed for user `57eef6228a1c4b7eaeb142dc4a74e4b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:21.327471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c468f02a-c75c-4b09-b1d3-1242dd415064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:21.356876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 879/1774] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:43:21.364447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da9ecb-cae8-4d3b-8c32-a7cf4670f6f7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:21.364603 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da9ecb-cae8-4d3b-8c32-a7cf4670f6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:21.364819 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da9ecb-cae8-4d3b-8c32-a7cf4670f6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:21.405823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:21.449446 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c4da9ecb-cae8-4d3b-8c32-a7cf4670f6f7 None None] MFA Rules not processed for user `38ea91bf04e741c793e4a06db9fb046f`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:21.451379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c4da9ecb-cae8-4d3b-8c32-a7cf4670f6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:21.594662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 896/1775] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:21.598806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468f02a-c75c-4b09-b1d3-1242dd415064 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:21.689300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 880/1776] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:21.776738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468f02a-c75c-4b09-b1d3-1242dd415064 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:21.792895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0d23b-094b-4f8d-8db5-0018854950cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:21.793829 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0d23b-094b-4f8d-8db5-0018854950cf admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:21.794258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0d23b-094b-4f8d-8db5-0018854950cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:21.794676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0d23b-094b-4f8d-8db5-0018854950cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:21.795460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0d23b-094b-4f8d-8db5-0018854950cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:21.802343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0d23b-094b-4f8d-8db5-0018854950cf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:21.865520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 897/1777] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:21 2022] POST /identity/v3/projects => generated 379 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:43:21.869328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da9ecb-cae8-4d3b-8c32-a7cf4670f6f7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:21.891950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da9ecb-cae8-4d3b-8c32-a7cf4670f6f7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:21.976622 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0d23b-094b-4f8d-8db5-0018854950cf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:22.024344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:22.046988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c891b-e17c-4fa8-b8e6-cfb0958f7969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:22.048006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c891b-e17c-4fa8-b8e6-cfb0958f7969 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:22.048315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c891b-e17c-4fa8-b8e6-cfb0958f7969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:22.048797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c891b-e17c-4fa8-b8e6-cfb0958f7969 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:22.049935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c891b-e17c-4fa8-b8e6-cfb0958f7969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:22.055917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c891b-e17c-4fa8-b8e6-cfb0958f7969 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:22.099145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 881/1778] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:21 2022] POST /identity/v3/users => generated 296 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:22.102909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c891b-e17c-4fa8-b8e6-cfb0958f7969 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:22.120318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0d23b-094b-4f8d-8db5-0018854950cf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:22.125598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c891b-e17c-4fa8-b8e6-cfb0958f7969 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:22.131757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbdec1-1e08-41e4-956c-66dda9976540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:22.132423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbdec1-1e08-41e4-956c-66dda9976540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:22.132736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbdec1-1e08-41e4-956c-66dda9976540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:22.133056 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbdec1-1e08-41e4-956c-66dda9976540 admin admin] PATH_INFO: `/v3/users/10d1b9314e994cd3add36f51051caa12` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:22.138126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbdec1-1e08-41e4-956c-66dda9976540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10d1b9314e994cd3add36f51051caa12)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:22.143161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbdec1-1e08-41e4-956c-66dda9976540 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:22.144806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bc52d-2b8e-49ed-ae19-a85041cb4bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:22.145441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bc52d-2b8e-49ed-ae19-a85041cb4bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:22.145723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bc52d-2b8e-49ed-ae19-a85041cb4bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:22.146076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bc52d-2b8e-49ed-ae19-a85041cb4bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:22.147002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760bc52d-2b8e-49ed-ae19-a85041cb4bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:22.148936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760bc52d-2b8e-49ed-ae19-a85041cb4bf9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:22.164317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 882/1779] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:22.170029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bc52d-2b8e-49ed-ae19-a85041cb4bf9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:22.193439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bc52d-2b8e-49ed-ae19-a85041cb4bf9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:22.211077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e88d6-ab02-4dbc-becc-aff4ee882b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:22.211892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e88d6-ab02-4dbc-becc-aff4ee882b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:22.212289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e88d6-ab02-4dbc-becc-aff4ee882b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:22.212709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e88d6-ab02-4dbc-becc-aff4ee882b20 admin admin] PATH_INFO: `/v3/projects/85ddcc2372ba4ef686e49e68aceb3477/users/b3a94a20089e4707a0a47f0777c39766/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:22.231429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:22.234637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b9cbdec1-1e08-41e4-956c-66dda9976540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10d1b9314e994cd3add36f51051caa12'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:22.249234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e88d6-ab02-4dbc-becc-aff4ee882b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ddcc2372ba4ef686e49e68aceb3477, user_id=b3a94a20089e4707a0a47f0777c39766, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:22.255914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e88d6-ab02-4dbc-becc-aff4ee882b20 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:22.270030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b9cbdec1-1e08-41e4-956c-66dda9976540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10d1b9314e994cd3add36f51051caa12'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:22.273592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b9cbdec1-1e08-41e4-956c-66dda9976540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10d1b9314e994cd3add36f51051caa12'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:22.275339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b9cbdec1-1e08-41e4-956c-66dda9976540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10d1b9314e994cd3add36f51051caa12'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:22.282733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 883/1780] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:22 2022] PUT /identity/v3/projects/85ddcc2372ba4ef686e49e68aceb3477/users/b3a94a20089e4707a0a47f0777c39766/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:22.287216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e88d6-ab02-4dbc-becc-aff4ee882b20 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:22.296091 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b9cbdec1-1e08-41e4-956c-66dda9976540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10d1b9314e994cd3add36f51051caa12'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:22.297137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2a6e88d6-ab02-4dbc-becc-aff4ee882b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:22.348463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 898/1781] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:21 2022] DELETE /identity/v3/users/10d1b9314e994cd3add36f51051caa12 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:22.350999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a112f6a-dd3b-4578-96f0-21b73ec590e2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:22.351354 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a112f6a-dd3b-4578-96f0-21b73ec590e2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:22.351710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a112f6a-dd3b-4578-96f0-21b73ec590e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:22.401777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0a112f6a-dd3b-4578-96f0-21b73ec590e2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:22.403952 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0a112f6a-dd3b-4578-96f0-21b73ec590e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:22.450870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e88d6-ab02-4dbc-becc-aff4ee882b20 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:22.613226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 899/1782] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:22.621684 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006999b9-8550-4246-9b6b-2b983ac954e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:22.622034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a112f6a-dd3b-4578-96f0-21b73ec590e2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:22.622398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006999b9-8550-4246-9b6b-2b983ac954e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:22.622585 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006999b9-8550-4246-9b6b-2b983ac954e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:22.622859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006999b9-8550-4246-9b6b-2b983ac954e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:22.623583 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006999b9-8550-4246-9b6b-2b983ac954e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:22.623907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0a112f6a-dd3b-4578-96f0-21b73ec590e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:22.625149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006999b9-8550-4246-9b6b-2b983ac954e8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:22.668069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 884/1783] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:22.673322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006999b9-8550-4246-9b6b-2b983ac954e8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:22.705229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006999b9-8550-4246-9b6b-2b983ac954e8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:22.716064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a37cfe-1e95-42b6-a258-80d1283c51fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:22.718751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37cfe-1e95-42b6-a258-80d1283c51fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:22.720889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37cfe-1e95-42b6-a258-80d1283c51fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:22.721343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37cfe-1e95-42b6-a258-80d1283c51fa admin admin] PATH_INFO: `/v3/projects/85ddcc2372ba4ef686e49e68aceb3477/users/b3a94a20089e4707a0a47f0777c39766/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:22.750034 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a37cfe-1e95-42b6-a258-80d1283c51fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ddcc2372ba4ef686e49e68aceb3477, user_id=b3a94a20089e4707a0a47f0777c39766, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:22.762583 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a37cfe-1e95-42b6-a258-80d1283c51fa admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:22.793110 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 885/1784] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:22 2022] PUT /identity/v3/projects/85ddcc2372ba4ef686e49e68aceb3477/users/b3a94a20089e4707a0a47f0777c39766/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:22.813649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909e6d5-097d-4a55-a74b-5002c3c21a89 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:22.813945 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909e6d5-097d-4a55-a74b-5002c3c21a89 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:22.814227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909e6d5-097d-4a55-a74b-5002c3c21a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:22.836648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a112f6a-dd3b-4578-96f0-21b73ec590e2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:22.901953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0909e6d5-097d-4a55-a74b-5002c3c21a89 None None] MFA Rules not processed for user `b3a94a20089e4707a0a47f0777c39766`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:22.903305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0909e6d5-097d-4a55-a74b-5002c3c21a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:22.994935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:23.087055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6578d-8be6-4e76-be4f-096aacc5b395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:23.090122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6578d-8be6-4e76-be4f-096aacc5b395 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:23.090553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6578d-8be6-4e76-be4f-096aacc5b395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:23.090862 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6578d-8be6-4e76-be4f-096aacc5b395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:23.193407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 886/1785] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:23.206224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:23.209469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909e6d5-097d-4a55-a74b-5002c3c21a89 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:23.211418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0909e6d5-097d-4a55-a74b-5002c3c21a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:23.288877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6578d-8be6-4e76-be4f-096aacc5b395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:23.290923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6578d-8be6-4e76-be4f-096aacc5b395 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:23.318356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 900/1786] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:43:23.328172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909e6d5-097d-4a55-a74b-5002c3c21a89 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:23.333451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cec8f4-8771-49e8-af65-dbc9db1e7621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:23.334091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec8f4-8771-49e8-af65-dbc9db1e7621 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:23.334378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec8f4-8771-49e8-af65-dbc9db1e7621 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:23.334662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec8f4-8771-49e8-af65-dbc9db1e7621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:23.348048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cec8f4-8771-49e8-af65-dbc9db1e7621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:23.349997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cec8f4-8771-49e8-af65-dbc9db1e7621 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:23.388702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 887/1787] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:24.264267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6578d-8be6-4e76-be4f-096aacc5b395 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:24.286302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6578d-8be6-4e76-be4f-096aacc5b395 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:24.326940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3107c-f753-4988-9b7e-3781cfb15185 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3a94a20089e4707a0a47f0777c39766', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85ddcc2372ba4ef686e49e68aceb3477', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:24.328687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3107c-f753-4988-9b7e-3781cfb15185 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:24.328993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3107c-f753-4988-9b7e-3781cfb15185 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:24.329275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3107c-f753-4988-9b7e-3781cfb15185 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:24.330124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3107c-f753-4988-9b7e-3781cfb15185 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:43:24.330616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3107c-f753-4988-9b7e-3781cfb15185 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:24.340613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3107c-f753-4988-9b7e-3781cfb15185 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:24.381491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 901/1788] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:43:24.383239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cec8f4-8771-49e8-af65-dbc9db1e7621 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:24.389030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3107c-f753-4988-9b7e-3781cfb15185 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:24.394514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-48d3107c-f753-4988-9b7e-3781cfb15185 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:24.398227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cec8f4-8771-49e8-af65-dbc9db1e7621 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:24.432845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bcb1a-3907-436f-a50c-57b9a1ba4ce6 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3a94a20089e4707a0a47f0777c39766', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85ddcc2372ba4ef686e49e68aceb3477', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:24.433490 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bcb1a-3907-436f-a50c-57b9a1ba4ce6 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:24.433780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bcb1a-3907-436f-a50c-57b9a1ba4ce6 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:24.434066 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bcb1a-3907-436f-a50c-57b9a1ba4ce6 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:24.434725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bcb1a-3907-436f-a50c-57b9a1ba4ce6 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:24.437809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3107c-f753-4988-9b7e-3781cfb15185 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:24.445145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bcb1a-3907-436f-a50c-57b9a1ba4ce6 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:24.447563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:24.448553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:24.448987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:24.449435 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] PATH_INFO: `/v3/projects/5e596987eeee4dce8eb19823b33954d4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:24.451557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e596987eeee4dce8eb19823b33954d4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:24.458785 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:24.525322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 888/1789] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:24 2022] POST /identity/v3/projects => generated 354 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:43:24.529110 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bcb1a-3907-436f-a50c-57b9a1ba4ce6 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:24.642488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e596987eeee4dce8eb19823b33954d4', 'request_id': 'req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:24.686980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e596987eeee4dce8eb19823b33954d4', 'request_id': 'req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:24.707454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e596987eeee4dce8eb19823b33954d4', 'request_id': 'req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:24.726570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bcb1a-3907-436f-a50c-57b9a1ba4ce6 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:24.732151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] The token cache is being invalidate because project 5e596987eeee4dce8eb19823b33954d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:24.732540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:24.733601 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:24.741245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 902/1790] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:24 2022] DELETE /identity/v3/projects/5e596987eeee4dce8eb19823b33954d4 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:24.742721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:24.744738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:24.827576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:24.861494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f3ca5-5c80-484f-bf4c-84c3e428a96f tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3a94a20089e4707a0a47f0777c39766', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85ddcc2372ba4ef686e49e68aceb3477', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:24.862124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3ca5-5c80-484f-bf4c-84c3e428a96f tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:24.862575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3ca5-5c80-484f-bf4c-84c3e428a96f tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:24.863199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3ca5-5c80-484f-bf4c-84c3e428a96f tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:24.863955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f3ca5-5c80-484f-bf4c-84c3e428a96f tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:43:24.864321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f3ca5-5c80-484f-bf4c-84c3e428a96f tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:24.868557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f3ca5-5c80-484f-bf4c-84c3e428a96f tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:24.905434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 889/1791] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:43:24.936176 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5db8e-bf2b-48fe-bf88-92f6f7167f2f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:24.954176 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:24.954912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:24.955197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:24.955470 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] PATH_INFO: `/v3/users/938b4eff73d1434cb0ffa591efee9d88` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:24.959377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=938b4eff73d1434cb0ffa591efee9d88)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:24.968357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:25.006692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f3ca5-5c80-484f-bf4c-84c3e428a96f tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:25.010978 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0f1f3ca5-5c80-484f-bf4c-84c3e428a96f tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:25.031030 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f3ca5-5c80-484f-bf4c-84c3e428a96f tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:25.068953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3a94a20089e4707a0a47f0777c39766', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85ddcc2372ba4ef686e49e68aceb3477', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:25.069827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:25.071738 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:25.072235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] PATH_INFO: `/v3/projects/be9a4722bc804156837f1bc400865270` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:25.075318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: Authorizing `identity:delete_project(project_id=be9a4722bc804156837f1bc400865270)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:25.087825 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:25.108265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:25.137200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '938b4eff73d1434cb0ffa591efee9d88'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:25.155635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '938b4eff73d1434cb0ffa591efee9d88'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:25.158675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '938b4eff73d1434cb0ffa591efee9d88'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:25.160398 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '938b4eff73d1434cb0ffa591efee9d88'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:25.187022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '938b4eff73d1434cb0ffa591efee9d88'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:25.217546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 903/1792] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:24 2022] DELETE /identity/v3/users/938b4eff73d1434cb0ffa591efee9d88 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:25.260163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be9a4722bc804156837f1bc400865270', 'request_id': 'req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:25.267915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be9a4722bc804156837f1bc400865270', 'request_id': 'req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:25.280592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be9a4722bc804156837f1bc400865270', 'request_id': 'req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:25.315011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] The token cache is being invalidate because project be9a4722bc804156837f1bc400865270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:25.315873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:25.317182 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:25.322220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 890/1793] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:25 2022] DELETE /identity/v3/projects/be9a4722bc804156837f1bc400865270 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:25.747988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:25.750066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:25.875110 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb099d9b-9c43-4b26-8c5c-75c639687229 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:25.876436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:25.878097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:25.937088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88651b96-ac25-4ec2-9cc9-9f289e1a96c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:25.937730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88651b96-ac25-4ec2-9cc9-9f289e1a96c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:25.938014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88651b96-ac25-4ec2-9cc9-9f289e1a96c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:25.938573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88651b96-ac25-4ec2-9cc9-9f289e1a96c4 admin admin] PATH_INFO: `/v3/users/57eef6228a1c4b7eaeb142dc4a74e4b1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:25.939443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88651b96-ac25-4ec2-9cc9-9f289e1a96c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57eef6228a1c4b7eaeb142dc4a74e4b1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:25.942213 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88651b96-ac25-4ec2-9cc9-9f289e1a96c4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:25.955589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623b0cf-3367-49a5-8ba1-dc9d06080dc9 tempest-VolumeQuotaClassesTest-597000311 tempest-VolumeQuotaClassesTest-597000311-project-admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:25.961683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:25.963007 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:25.963389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:25.963696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] PATH_INFO: `/v3/projects/359313eab1a647d2946ff77a7c3e4632` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:25.964954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=359313eab1a647d2946ff77a7c3e4632)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:25.967218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:25.992871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-88651b96-ac25-4ec2-9cc9-9f289e1a96c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57eef6228a1c4b7eaeb142dc4a74e4b1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.000326 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-88651b96-ac25-4ec2-9cc9-9f289e1a96c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57eef6228a1c4b7eaeb142dc4a74e4b1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.001058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-88651b96-ac25-4ec2-9cc9-9f289e1a96c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57eef6228a1c4b7eaeb142dc4a74e4b1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.001756 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-88651b96-ac25-4ec2-9cc9-9f289e1a96c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57eef6228a1c4b7eaeb142dc4a74e4b1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.010057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-88651b96-ac25-4ec2-9cc9-9f289e1a96c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57eef6228a1c4b7eaeb142dc4a74e4b1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.014790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:26.021716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '359313eab1a647d2946ff77a7c3e4632', 'request_id': 'req-017e3718-1bd0-4433-80bd-623c11ccd163'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.027891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '359313eab1a647d2946ff77a7c3e4632', 'request_id': 'req-017e3718-1bd0-4433-80bd-623c11ccd163'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.031591 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 904/1794] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:25 2022] DELETE /identity/v3/users/57eef6228a1c4b7eaeb142dc4a74e4b1 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:26.034800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '359313eab1a647d2946ff77a7c3e4632', 'request_id': 'req-017e3718-1bd0-4433-80bd-623c11ccd163'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.037091 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c280-1f8f-4e29-ba2e-dabc2f6d0fc3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.037387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c280-1f8f-4e29-ba2e-dabc2f6d0fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.037662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c280-1f8f-4e29-ba2e-dabc2f6d0fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.037763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:26.049440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] The token cache is being invalidate because project 359313eab1a647d2946ff77a7c3e4632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:26.050565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.051368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-017e3718-1bd0-4433-80bd-623c11ccd163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.053148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 891/1795] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:25 2022] DELETE /identity/v3/projects/359313eab1a647d2946ff77a7c3e4632 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:26.057861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b33983-c713-4927-b456-bf30ad7a88ae None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.058006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b680c280-1f8f-4e29-ba2e-dabc2f6d0fc3 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:26.058153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b33983-c713-4927-b456-bf30ad7a88ae None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.058494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b33983-c713-4927-b456-bf30ad7a88ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.059149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b680c280-1f8f-4e29-ba2e-dabc2f6d0fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:26.079194 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-54b33983-c713-4927-b456-bf30ad7a88ae None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:26.080411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-54b33983-c713-4927-b456-bf30ad7a88ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:26.125914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 905/1796] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:26.135969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680c280-1f8f-4e29-ba2e-dabc2f6d0fc3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:26.137491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b680c280-1f8f-4e29-ba2e-dabc2f6d0fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:26.145827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 892/1797] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:26.153446 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547034d-c6b1-4eff-9122-03f20a98147e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.153734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547034d-c6b1-4eff-9122-03f20a98147e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.154011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547034d-c6b1-4eff-9122-03f20a98147e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.173342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8547034d-c6b1-4eff-9122-03f20a98147e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:26.174543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8547034d-c6b1-4eff-9122-03f20a98147e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:26.184941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 893/1798] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:26.189141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547034d-c6b1-4eff-9122-03f20a98147e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:26.196585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680c280-1f8f-4e29-ba2e-dabc2f6d0fc3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:26.200149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547034d-c6b1-4eff-9122-03f20a98147e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:26.200372 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fb7ab-4328-4620-ab95-6c4908f64cfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:26.201016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb7ab-4328-4620-ab95-6c4908f64cfa service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.201297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb7ab-4328-4620-ab95-6c4908f64cfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.201567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb7ab-4328-4620-ab95-6c4908f64cfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.204877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542c94a-6cc4-4dac-8b71-c30fa4b40fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:26.205508 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542c94a-6cc4-4dac-8b71-c30fa4b40fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.205793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542c94a-6cc4-4dac-8b71-c30fa4b40fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.206595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542c94a-6cc4-4dac-8b71-c30fa4b40fe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.207646 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542c94a-6cc4-4dac-8b71-c30fa4b40fe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:43:26.208143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542c94a-6cc4-4dac-8b71-c30fa4b40fe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:26.210536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542c94a-6cc4-4dac-8b71-c30fa4b40fe6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:26.211207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fb7ab-4328-4620-ab95-6c4908f64cfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:26.212676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fb7ab-4328-4620-ab95-6c4908f64cfa service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:26.226857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 894/1799] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:43:26.227614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 906/1800] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:43:26.229348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542c94a-6cc4-4dac-8b71-c30fa4b40fe6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:26.242770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542c94a-6cc4-4dac-8b71-c30fa4b40fe6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:26.247833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20975a6f-b4ec-48b9-a572-344d21bde7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:26.248455 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20975a6f-b4ec-48b9-a572-344d21bde7fa admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.248950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20975a6f-b4ec-48b9-a572-344d21bde7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.249233 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20975a6f-b4ec-48b9-a572-344d21bde7fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.249876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20975a6f-b4ec-48b9-a572-344d21bde7fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:26.251862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20975a6f-b4ec-48b9-a572-344d21bde7fa admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:26.295567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 895/1801] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:26 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:26.300576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fb7ab-4328-4620-ab95-6c4908f64cfa service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:26.337656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fb7ab-4328-4620-ab95-6c4908f64cfa service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:26.422676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08815bd-b294-43d8-baa0-cf539af149b1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.422987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08815bd-b294-43d8-baa0-cf539af149b1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.423746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08815bd-b294-43d8-baa0-cf539af149b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.534414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fd49f-5b22-45ff-8d24-0039f31cabc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:26.535057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fd49f-5b22-45ff-8d24-0039f31cabc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.535345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fd49f-5b22-45ff-8d24-0039f31cabc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.535624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fd49f-5b22-45ff-8d24-0039f31cabc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.536270 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fd49f-5b22-45ff-8d24-0039f31cabc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:26.554860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fd49f-5b22-45ff-8d24-0039f31cabc8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:26.590071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b08815bd-b294-43d8-baa0-cf539af149b1 None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:26.591325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b08815bd-b294-43d8-baa0-cf539af149b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:26.623384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 907/1802] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:26 2022] POST /identity/v3/users => generated 292 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:26.626683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fd49f-5b22-45ff-8d24-0039f31cabc8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:26.643466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fd49f-5b22-45ff-8d24-0039f31cabc8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:26.648979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c914a-9dae-4749-a7d9-8c355edc681f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:26.649965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c914a-9dae-4749-a7d9-8c355edc681f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.650339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c914a-9dae-4749-a7d9-8c355edc681f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.650645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c914a-9dae-4749-a7d9-8c355edc681f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.651487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c914a-9dae-4749-a7d9-8c355edc681f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:26.653311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c914a-9dae-4749-a7d9-8c355edc681f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:26.671481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 908/1803] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:26.677100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c914a-9dae-4749-a7d9-8c355edc681f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:26.704014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c914a-9dae-4749-a7d9-8c355edc681f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:26.711030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53593ff3-b4f3-4452-b071-61de6c28d668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:26.714557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53593ff3-b4f3-4452-b071-61de6c28d668 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.714557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53593ff3-b4f3-4452-b071-61de6c28d668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.714557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53593ff3-b4f3-4452-b071-61de6c28d668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.714557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53593ff3-b4f3-4452-b071-61de6c28d668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:26.715414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53593ff3-b4f3-4452-b071-61de6c28d668 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:26.717607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:26.751568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 909/1804] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:26.751568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53593ff3-b4f3-4452-b071-61de6c28d668 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:26.754835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-53593ff3-b4f3-4452-b071-61de6c28d668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:26.766692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 896/1805] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:43:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:26.767636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08815bd-b294-43d8-baa0-cf539af149b1 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:26.787380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53593ff3-b4f3-4452-b071-61de6c28d668 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:26.802093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08815bd-b294-43d8-baa0-cf539af149b1 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:26.805200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:26.805863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.806139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.806409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] PATH_INFO: `/v3/projects/aad73309590d4d58befa7d7258839dda` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.808377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad73309590d4d58befa7d7258839dda)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:26.812365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:26.818274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdc45c-f993-40ec-826a-1bef7c565c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:26.818975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc45c-f993-40ec-826a-1bef7c565c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.819297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc45c-f993-40ec-826a-1bef7c565c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.819670 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc45c-f993-40ec-826a-1bef7c565c35 admin admin] PATH_INFO: `/v3/projects/d64df612870b46eb82b54cb7aa2c2adc/users/969ab2b80ca047b3be973459644a45d1/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.850380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdc45c-f993-40ec-826a-1bef7c565c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64df612870b46eb82b54cb7aa2c2adc, user_id=969ab2b80ca047b3be973459644a45d1, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:26.854817 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdc45c-f993-40ec-826a-1bef7c565c35 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:26.872112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:26.895481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 897/1806] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:26 2022] PUT /identity/v3/projects/d64df612870b46eb82b54cb7aa2c2adc/users/969ab2b80ca047b3be973459644a45d1/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:26.897494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1e120-5e76-47e6-9b22-27179511f66a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:26.897977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1e120-5e76-47e6-9b22-27179511f66a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:26.898389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1e120-5e76-47e6-9b22-27179511f66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:26.931521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad73309590d4d58befa7d7258839dda', 'request_id': 'req-313cb40d-b303-4fff-bce3-986104b088c7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.955091 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad73309590d4d58befa7d7258839dda', 'request_id': 'req-313cb40d-b303-4fff-bce3-986104b088c7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.970057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad73309590d4d58befa7d7258839dda', 'request_id': 'req-313cb40d-b303-4fff-bce3-986104b088c7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.970498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e4b1e120-5e76-47e6-9b22-27179511f66a None None] MFA Rules not processed for user `969ab2b80ca047b3be973459644a45d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:26.973542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e4b1e120-5e76-47e6-9b22-27179511f66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:26.998051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] The token cache is being invalidate because project aad73309590d4d58befa7d7258839dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:26.998051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:26.998051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:27.000804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 910/1807] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:26 2022] DELETE /identity/v3/projects/aad73309590d4d58befa7d7258839dda => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:27.005386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:27.007758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:27.199084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cb40d-b303-4fff-bce3-986104b088c7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:27.214416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:27.215286 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:27.215665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:27.216040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] PATH_INFO: `/v3/users/b3a94a20089e4707a0a47f0777c39766` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:27.216641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 898/1808] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:27.219116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a94a20089e4707a0a47f0777c39766)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:27.221395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:27.229129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1e120-5e76-47e6-9b22-27179511f66a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:27.236698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e4b1e120-5e76-47e6-9b22-27179511f66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:27.347418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a94a20089e4707a0a47f0777c39766'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:27.369810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a94a20089e4707a0a47f0777c39766'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:27.371495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a94a20089e4707a0a47f0777c39766'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:27.373527 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a94a20089e4707a0a47f0777c39766'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:27.396057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a94a20089e4707a0a47f0777c39766'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:27.445437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:27.462159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 911/1809] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:27 2022] DELETE /identity/v3/users/b3a94a20089e4707a0a47f0777c39766 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:27.463583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:27.469094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:27.538304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1e120-5e76-47e6-9b22-27179511f66a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:27.620054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2b793-02fa-4ba8-bc6f-eb3fb6e20884 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:27.630890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc9693-d20b-4a09-a3ba-5272c1643558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:27.632840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc9693-d20b-4a09-a3ba-5272c1643558 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:27.633132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc9693-d20b-4a09-a3ba-5272c1643558 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:27.633409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc9693-d20b-4a09-a3ba-5272c1643558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:27.642530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-95fc9693-d20b-4a09-a3ba-5272c1643558 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:27.735932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24778f2-f6e4-4db9-8277-7472a08c965c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:27.736593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24778f2-f6e4-4db9-8277-7472a08c965c service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:27.736876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24778f2-f6e4-4db9-8277-7472a08c965c service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:27.737152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24778f2-f6e4-4db9-8277-7472a08c965c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:27.742908 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d24778f2-f6e4-4db9-8277-7472a08c965c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:27.773943 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc9693-d20b-4a09-a3ba-5272c1643558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:27.776860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc9693-d20b-4a09-a3ba-5272c1643558 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:27.777656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:27.816554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 912/1810] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:27.907397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24778f2-f6e4-4db9-8277-7472a08c965c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:27.910263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24778f2-f6e4-4db9-8277-7472a08c965c service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:27.931532 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 899/1811] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:43:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:43:30.123686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c72c0-0295-4757-8c04-5c032e75641b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:30.132207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c72c0-0295-4757-8c04-5c032e75641b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:30.132207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c72c0-0295-4757-8c04-5c032e75641b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:30.186350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-574c72c0-0295-4757-8c04-5c032e75641b None None] MFA Rules not processed for user `30546ba7f5614f878a9ba322c7c55ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:30.187840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-574c72c0-0295-4757-8c04-5c032e75641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:30.375269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 913/1812] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Aug 31 11:43:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:30.481417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24778f2-f6e4-4db9-8277-7472a08c965c service swift] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:30.483719 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d24778f2-f6e4-4db9-8277-7472a08c965c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:30.513828 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24778f2-f6e4-4db9-8277-7472a08c965c service swift] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:30.530949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efda289-c09d-49c1-9682-05e13a5f9706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:30.531591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efda289-c09d-49c1-9682-05e13a5f9706 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:30.531884 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efda289-c09d-49c1-9682-05e13a5f9706 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:30.532187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efda289-c09d-49c1-9682-05e13a5f9706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:30.565906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efda289-c09d-49c1-9682-05e13a5f9706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:30.567489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efda289-c09d-49c1-9682-05e13a5f9706 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:30.596580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 900/1813] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:31.586199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c72c0-0295-4757-8c04-5c032e75641b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:31.587610 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-574c72c0-0295-4757-8c04-5c032e75641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:31.767960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c72c0-0295-4757-8c04-5c032e75641b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:31.782345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:31.783047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:31.783330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:31.783603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] PATH_INFO: `/v3/projects/85ddcc2372ba4ef686e49e68aceb3477` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:31.787733 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85ddcc2372ba4ef686e49e68aceb3477)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:31.795577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:31.799310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:31.941295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85ddcc2372ba4ef686e49e68aceb3477', 'request_id': 'req-7338031c-b3b5-41a7-8978-42e7ded89550'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:31.956432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85ddcc2372ba4ef686e49e68aceb3477', 'request_id': 'req-7338031c-b3b5-41a7-8978-42e7ded89550'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:31.988069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85ddcc2372ba4ef686e49e68aceb3477', 'request_id': 'req-7338031c-b3b5-41a7-8978-42e7ded89550'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:32.028309 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] The token cache is being invalidate because project 85ddcc2372ba4ef686e49e68aceb3477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:32.029099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:32.033502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7338031c-b3b5-41a7-8978-42e7ded89550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:32.040829 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 914/1814] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:31 2022] DELETE /identity/v3/projects/85ddcc2372ba4ef686e49e68aceb3477 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:32.042294 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb660428-d8f5-402e-9cb6-4c026e670a85 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:32.042786 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb660428-d8f5-402e-9cb6-4c026e670a85 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:32.042893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb660428-d8f5-402e-9cb6-4c026e670a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:32.083736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-eb660428-d8f5-402e-9cb6-4c026e670a85 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:32.085084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-eb660428-d8f5-402e-9cb6-4c026e670a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:32.262686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 901/1815] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:32.270844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367976d4-a77b-462f-b9c3-600a572b50d9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:32.271141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367976d4-a77b-462f-b9c3-600a572b50d9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:32.271415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367976d4-a77b-462f-b9c3-600a572b50d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:32.317310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-367976d4-a77b-462f-b9c3-600a572b50d9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:32.323357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-367976d4-a77b-462f-b9c3-600a572b50d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:32.349700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 915/1816] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:32.356455 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb660428-d8f5-402e-9cb6-4c026e670a85 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:32.369831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb660428-d8f5-402e-9cb6-4c026e670a85 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:32.377535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd9490-1aea-40ed-a822-a33fa7a8e5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:32.378217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd9490-1aea-40ed-a822-a33fa7a8e5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:32.378490 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd9490-1aea-40ed-a822-a33fa7a8e5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:32.379030 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd9490-1aea-40ed-a822-a33fa7a8e5c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:32.380796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd9490-1aea-40ed-a822-a33fa7a8e5c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:43:32.381422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd9490-1aea-40ed-a822-a33fa7a8e5c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:32.383814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd9490-1aea-40ed-a822-a33fa7a8e5c4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:32.403177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 902/1817] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:43:32.409444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367976d4-a77b-462f-b9c3-600a572b50d9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:32.438739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367976d4-a77b-462f-b9c3-600a572b50d9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:32.455881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cc6e6-574a-4ba0-8a08-846e132b2998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:32.456738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cc6e6-574a-4ba0-8a08-846e132b2998 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:32.457205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cc6e6-574a-4ba0-8a08-846e132b2998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:32.457766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cc6e6-574a-4ba0-8a08-846e132b2998 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:32.458511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cc6e6-574a-4ba0-8a08-846e132b2998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:32.465556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cc6e6-574a-4ba0-8a08-846e132b2998 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:32.538618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 916/1818] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:32 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:32.542905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd9490-1aea-40ed-a822-a33fa7a8e5c4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:32.576439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd9490-1aea-40ed-a822-a33fa7a8e5c4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:32.751138 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb6f67-5e3a-4ec0-a234-73ba003cb1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:32.751286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb6f67-5e3a-4ec0-a234-73ba003cb1dc admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:32.751576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb6f67-5e3a-4ec0-a234-73ba003cb1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:32.751903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb6f67-5e3a-4ec0-a234-73ba003cb1dc admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:32.752611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb6f67-5e3a-4ec0-a234-73ba003cb1dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:32.755672 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb6f67-5e3a-4ec0-a234-73ba003cb1dc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:32.797345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 903/1819] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:32 2022] POST /identity/v3/users => generated 292 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:32.801816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cc6e6-574a-4ba0-8a08-846e132b2998 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:32.822576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cc6e6-574a-4ba0-8a08-846e132b2998 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:32.834732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcad35-da1a-48cf-9cd1-d2551d7b8d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:32.835790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcad35-da1a-48cf-9cd1-d2551d7b8d85 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:32.836155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcad35-da1a-48cf-9cd1-d2551d7b8d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:32.836673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcad35-da1a-48cf-9cd1-d2551d7b8d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:32.837500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcad35-da1a-48cf-9cd1-d2551d7b8d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:32.841165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcad35-da1a-48cf-9cd1-d2551d7b8d85 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:32.882842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 917/1820] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:32.887117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb6f67-5e3a-4ec0-a234-73ba003cb1dc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:32.905143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb6f67-5e3a-4ec0-a234-73ba003cb1dc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:32.937146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e260f1c-6318-4909-b66f-b03ef4bb5fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:32.937790 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e260f1c-6318-4909-b66f-b03ef4bb5fef admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:32.939558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e260f1c-6318-4909-b66f-b03ef4bb5fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:32.939849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e260f1c-6318-4909-b66f-b03ef4bb5fef admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:32.941999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e260f1c-6318-4909-b66f-b03ef4bb5fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:32.950043 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e260f1c-6318-4909-b66f-b03ef4bb5fef admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:32.990707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 904/1821] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:32.995683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcad35-da1a-48cf-9cd1-d2551d7b8d85 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:33.015493 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcad35-da1a-48cf-9cd1-d2551d7b8d85 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:33.045011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e1ef2-fedd-49c1-a331-9478389f327c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:33.055749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e1ef2-fedd-49c1-a331-9478389f327c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:33.056128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e1ef2-fedd-49c1-a331-9478389f327c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:33.056312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e1ef2-fedd-49c1-a331-9478389f327c admin admin] PATH_INFO: `/v3/projects/116794d91ff8488f858ac0aa71e3e3ed/users/13497bcc95bd4c929bbe0e889410fe8a/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:33.099394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e1ef2-fedd-49c1-a331-9478389f327c admin admin] RBAC: Authorizing `identity:create_grant(project_id=116794d91ff8488f858ac0aa71e3e3ed, user_id=13497bcc95bd4c929bbe0e889410fe8a, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:33.104528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e1ef2-fedd-49c1-a331-9478389f327c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:33.138422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 918/1822] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:32 2022] PUT /identity/v3/projects/116794d91ff8488f858ac0aa71e3e3ed/users/13497bcc95bd4c929bbe0e889410fe8a/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:33.152752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b0482-fb9a-4a8d-8b3e-b6fcf25faf63 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:33.152752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b0482-fb9a-4a8d-8b3e-b6fcf25faf63 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:33.152752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b0482-fb9a-4a8d-8b3e-b6fcf25faf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:33.187010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:33.225002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-860b0482-fb9a-4a8d-8b3e-b6fcf25faf63 None None] MFA Rules not processed for user `13497bcc95bd4c929bbe0e889410fe8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:33.227911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-860b0482-fb9a-4a8d-8b3e-b6fcf25faf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:33.403181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 905/1823] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:33.408286 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e1ef2-fedd-49c1-a331-9478389f327c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:33.580207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e1ef2-fedd-49c1-a331-9478389f327c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:33.594629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532fcff-062a-4447-aa42-6e80848f6839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:33.595405 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532fcff-062a-4447-aa42-6e80848f6839 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:33.595791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532fcff-062a-4447-aa42-6e80848f6839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:33.596165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532fcff-062a-4447-aa42-6e80848f6839 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:33.597521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532fcff-062a-4447-aa42-6e80848f6839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:33.599574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532fcff-062a-4447-aa42-6e80848f6839 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:33.678335 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 919/1824] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:33 2022] POST /identity/v3/projects => generated 383 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:33.682230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b0482-fb9a-4a8d-8b3e-b6fcf25faf63 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:33.696956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b0482-fb9a-4a8d-8b3e-b6fcf25faf63 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:33.785075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368a357-94c0-4a34-abce-7e23cd767635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:33.785075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368a357-94c0-4a34-abce-7e23cd767635 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:33.785075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368a357-94c0-4a34-abce-7e23cd767635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:33.785075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368a357-94c0-4a34-abce-7e23cd767635 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:33.785570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9368a357-94c0-4a34-abce-7e23cd767635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:33.785570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9368a357-94c0-4a34-abce-7e23cd767635 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:33.817648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 906/1825] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:33 2022] POST /identity/v3/users => generated 298 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:33.821588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532fcff-062a-4447-aa42-6e80848f6839 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:33.842600 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532fcff-062a-4447-aa42-6e80848f6839 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:33.870663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210c449-8195-437b-8148-a07a22135a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:33.871426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210c449-8195-437b-8148-a07a22135a2a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:33.871811 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210c449-8195-437b-8148-a07a22135a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:33.872184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210c449-8195-437b-8148-a07a22135a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:33.873001 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210c449-8195-437b-8148-a07a22135a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:33.874700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210c449-8195-437b-8148-a07a22135a2a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:33.896560 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 920/1826] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:33.900892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368a357-94c0-4a34-abce-7e23cd767635 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:33.915353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368a357-94c0-4a34-abce-7e23cd767635 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:33.926570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a52c7-f5c3-47b9-8406-5051fd2150c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:33.928209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a52c7-f5c3-47b9-8406-5051fd2150c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:33.931198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a52c7-f5c3-47b9-8406-5051fd2150c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:33.931569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a52c7-f5c3-47b9-8406-5051fd2150c7 admin admin] PATH_INFO: `/v3/projects/502eac5866d043a4b56be279d1f13fc8/users/ccaa9ea4ec3640889f7173e586c0eaf0/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:33.998598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a52c7-f5c3-47b9-8406-5051fd2150c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=502eac5866d043a4b56be279d1f13fc8, user_id=ccaa9ea4ec3640889f7173e586c0eaf0, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:34.002273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a52c7-f5c3-47b9-8406-5051fd2150c7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:34.022062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 907/1827] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:33 2022] PUT /identity/v3/projects/502eac5866d043a4b56be279d1f13fc8/users/ccaa9ea4ec3640889f7173e586c0eaf0/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:34.026122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210c449-8195-437b-8148-a07a22135a2a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:34.033398 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:34.038912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffacca1-9913-48cc-9738-784e81052991 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:34.039204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffacca1-9913-48cc-9738-784e81052991 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:34.039480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffacca1-9913-48cc-9738-784e81052991 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:34.039764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210c449-8195-437b-8148-a07a22135a2a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:34.041000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 908/1828] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:43:34.064769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c045d0c-162f-4084-b9dc-a809018a4091 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:34.065065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c045d0c-162f-4084-b9dc-a809018a4091 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:34.066546 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c045d0c-162f-4084-b9dc-a809018a4091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:34.090833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3c045d0c-162f-4084-b9dc-a809018a4091 None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:34.092253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3c045d0c-162f-4084-b9dc-a809018a4091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:34.138419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291abf4-1fec-425f-89d4-36589be7b62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:34.139141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291abf4-1fec-425f-89d4-36589be7b62a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:34.139488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291abf4-1fec-425f-89d4-36589be7b62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:34.139824 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291abf4-1fec-425f-89d4-36589be7b62a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:34.140554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291abf4-1fec-425f-89d4-36589be7b62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:34.142246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291abf4-1fec-425f-89d4-36589be7b62a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:34.153816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 921/1829] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:34.158930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291abf4-1fec-425f-89d4-36589be7b62a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:34.163923 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:34.183082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291abf4-1fec-425f-89d4-36589be7b62a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:34.186674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 909/1830] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:43:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:34.194912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654dcb0-2b7e-46c0-840d-28872aa9fb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:34.195780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654dcb0-2b7e-46c0-840d-28872aa9fb0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:34.196158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654dcb0-2b7e-46c0-840d-28872aa9fb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:34.196631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654dcb0-2b7e-46c0-840d-28872aa9fb0c admin admin] PATH_INFO: `/v3/projects/502eac5866d043a4b56be279d1f13fc8/users/ccaa9ea4ec3640889f7173e586c0eaf0/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:34.199006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654dcb0-2b7e-46c0-840d-28872aa9fb0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=502eac5866d043a4b56be279d1f13fc8, user_id=ccaa9ea4ec3640889f7173e586c0eaf0, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:34.202052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c045d0c-162f-4084-b9dc-a809018a4091 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:34.205612 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654dcb0-2b7e-46c0-840d-28872aa9fb0c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:34.209323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3c045d0c-162f-4084-b9dc-a809018a4091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:34.223812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 922/1831] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:34 2022] PUT /identity/v3/projects/502eac5866d043a4b56be279d1f13fc8/users/ccaa9ea4ec3640889f7173e586c0eaf0/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:34.233016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c045d0c-162f-4084-b9dc-a809018a4091 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:34.234215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa592cf2-1fde-428e-8fc6-ac0c1278586c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:34.234503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa592cf2-1fde-428e-8fc6-ac0c1278586c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:34.234779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa592cf2-1fde-428e-8fc6-ac0c1278586c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:34.267398 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-aa592cf2-1fde-428e-8fc6-ac0c1278586c None None] MFA Rules not processed for user `ccaa9ea4ec3640889f7173e586c0eaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:34.268685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aa592cf2-1fde-428e-8fc6-ac0c1278586c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:34.316406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cdba6-4524-4602-947c-48abf82c0a3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:34.317092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cdba6-4524-4602-947c-48abf82c0a3b service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:34.317372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cdba6-4524-4602-947c-48abf82c0a3b service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:34.317641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cdba6-4524-4602-947c-48abf82c0a3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:34.328328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78cdba6-4524-4602-947c-48abf82c0a3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:34.329957 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78cdba6-4524-4602-947c-48abf82c0a3b service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:34.340776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 910/1832] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:43:34.354567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 923/1833] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:34.366103 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cdba6-4524-4602-947c-48abf82c0a3b service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:34.367529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e78cdba6-4524-4602-947c-48abf82c0a3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:34.439318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cdba6-4524-4602-947c-48abf82c0a3b service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:34.443499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7cf37-64a0-4ebc-9890-4e5ec3c6a495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:34.444104 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7cf37-64a0-4ebc-9890-4e5ec3c6a495 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:34.444382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7cf37-64a0-4ebc-9890-4e5ec3c6a495 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:34.444690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7cf37-64a0-4ebc-9890-4e5ec3c6a495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:34.453327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7cf37-64a0-4ebc-9890-4e5ec3c6a495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:34.454757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7cf37-64a0-4ebc-9890-4e5ec3c6a495 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:34.464269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 911/1834] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:34.571724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa592cf2-1fde-428e-8fc6-ac0c1278586c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:34.573058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aa592cf2-1fde-428e-8fc6-ac0c1278586c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:34.584893 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa592cf2-1fde-428e-8fc6-ac0c1278586c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:34.588455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee370ca-b068-44d2-8835-c8b1252777e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:34.589083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee370ca-b068-44d2-8835-c8b1252777e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:34.589368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee370ca-b068-44d2-8835-c8b1252777e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:34.589638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee370ca-b068-44d2-8835-c8b1252777e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:34.646069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee370ca-b068-44d2-8835-c8b1252777e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:34.647499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee370ca-b068-44d2-8835-c8b1252777e6 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:34.656281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 924/1835] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:38.652320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7cf37-64a0-4ebc-9890-4e5ec3c6a495 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:38.817865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7cf37-64a0-4ebc-9890-4e5ec3c6a495 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:38.842891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:38.843665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:38.843935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:38.844214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] PATH_INFO: `/v3/users/13497bcc95bd4c929bbe0e889410fe8a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:38.847535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13497bcc95bd4c929bbe0e889410fe8a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:38.849567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:38.925813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:39.003959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13497bcc95bd4c929bbe0e889410fe8a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:39.005324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13497bcc95bd4c929bbe0e889410fe8a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:39.032241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13497bcc95bd4c929bbe0e889410fe8a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:39.052596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13497bcc95bd4c929bbe0e889410fe8a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:39.053413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13497bcc95bd4c929bbe0e889410fe8a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:39.116972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 912/1836] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:38 2022] DELETE /identity/v3/users/13497bcc95bd4c929bbe0e889410fe8a => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:39.121323 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583edb9-aff5-412a-b8a6-c1a41fd75ffc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:39.121841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583edb9-aff5-412a-b8a6-c1a41fd75ffc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:39.121958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583edb9-aff5-412a-b8a6-c1a41fd75ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:39.147626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f583edb9-aff5-412a-b8a6-c1a41fd75ffc None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:39.149120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f583edb9-aff5-412a-b8a6-c1a41fd75ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:39.154071 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:39.254774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 925/1837] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:39.261530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:39.263258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:39.376576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e707c-e6d5-434d-a6b2-1ea00de87a52 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:39.381327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ec6d0-a3b5-4b1e-b692-d26ffeb5a524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:39.381938 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ec6d0-a3b5-4b1e-b692-d26ffeb5a524 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:39.382259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ec6d0-a3b5-4b1e-b692-d26ffeb5a524 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:39.382548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ec6d0-a3b5-4b1e-b692-d26ffeb5a524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:39.393800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ec6d0-a3b5-4b1e-b692-d26ffeb5a524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:39.395249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ec6d0-a3b5-4b1e-b692-d26ffeb5a524 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:39.408162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 913/1838] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:39.823194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583edb9-aff5-412a-b8a6-c1a41fd75ffc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:39.824692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f583edb9-aff5-412a-b8a6-c1a41fd75ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:39.838559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583edb9-aff5-412a-b8a6-c1a41fd75ffc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:39.847078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:39.848248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:39.848734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:39.849163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] PATH_INFO: `/v3/projects/116794d91ff8488f858ac0aa71e3e3ed` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:39.850606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=116794d91ff8488f858ac0aa71e3e3ed)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:39.853005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:39.930700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116794d91ff8488f858ac0aa71e3e3ed', 'request_id': 'req-5901f0a5-dcab-4037-9802-e99e7ba60a43'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:39.937699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116794d91ff8488f858ac0aa71e3e3ed', 'request_id': 'req-5901f0a5-dcab-4037-9802-e99e7ba60a43'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:39.944682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116794d91ff8488f858ac0aa71e3e3ed', 'request_id': 'req-5901f0a5-dcab-4037-9802-e99e7ba60a43'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:39.962878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] The token cache is being invalidate because project 116794d91ff8488f858ac0aa71e3e3ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:39.963221 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:39.964070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:39.966338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 926/1839] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:39 2022] DELETE /identity/v3/projects/116794d91ff8488f858ac0aa71e3e3ed => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:39.970550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ec6d0-a3b5-4b1e-b692-d26ffeb5a524 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:39.990872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ec6d0-a3b5-4b1e-b692-d26ffeb5a524 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:40.084507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0651b1c-ce11-4f48-beef-3d5b4b691274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:40.085358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0651b1c-ce11-4f48-beef-3d5b4b691274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:40.085643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0651b1c-ce11-4f48-beef-3d5b4b691274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:40.085915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0651b1c-ce11-4f48-beef-3d5b4b691274 admin admin] PATH_INFO: `/v3/users/ccaa9ea4ec3640889f7173e586c0eaf0` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:40.087577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0651b1c-ce11-4f48-beef-3d5b4b691274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccaa9ea4ec3640889f7173e586c0eaf0)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:40.094211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0651b1c-ce11-4f48-beef-3d5b4b691274 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:40.115182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:40.117942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:40.135600 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901f0a5-dcab-4037-9802-e99e7ba60a43 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:40.141494 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:40.142260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:40.142608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:40.142951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] PATH_INFO: `/v3/users/969ab2b80ca047b3be973459644a45d1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:40.145412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=969ab2b80ca047b3be973459644a45d1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:40.148320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:40.165543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:40.169239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f0651b1c-ce11-4f48-beef-3d5b4b691274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccaa9ea4ec3640889f7173e586c0eaf0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.170754 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f0651b1c-ce11-4f48-beef-3d5b4b691274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccaa9ea4ec3640889f7173e586c0eaf0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.180219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f0651b1c-ce11-4f48-beef-3d5b4b691274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccaa9ea4ec3640889f7173e586c0eaf0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.192165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f0651b1c-ce11-4f48-beef-3d5b4b691274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccaa9ea4ec3640889f7173e586c0eaf0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.193350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f0651b1c-ce11-4f48-beef-3d5b4b691274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccaa9ea4ec3640889f7173e586c0eaf0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.219489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 914/1840] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:39 2022] DELETE /identity/v3/users/ccaa9ea4ec3640889f7173e586c0eaf0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:40.236856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '969ab2b80ca047b3be973459644a45d1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.269895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '969ab2b80ca047b3be973459644a45d1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.272890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '969ab2b80ca047b3be973459644a45d1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.273559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '969ab2b80ca047b3be973459644a45d1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.287966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '969ab2b80ca047b3be973459644a45d1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.325092 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 927/1841] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:40 2022] DELETE /identity/v3/users/969ab2b80ca047b3be973459644a45d1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:40.329785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30edb-a51d-4f5a-b7b0-a3090545dc52 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:40.330120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30edb-a51d-4f5a-b7b0-a3090545dc52 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:40.330403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30edb-a51d-4f5a-b7b0-a3090545dc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:40.360466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e0e30edb-a51d-4f5a-b7b0-a3090545dc52 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:40.361968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e0e30edb-a51d-4f5a-b7b0-a3090545dc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:40.451325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 915/1842] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:40.477476 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:40.480123 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:40.492358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:40.633968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc988ca2-c92c-4779-ad68-ad020452ee06 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:40.657396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661231e1-6726-4a6c-9941-d78ede219b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:40.657396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661231e1-6726-4a6c-9941-d78ede219b64 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:40.657396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661231e1-6726-4a6c-9941-d78ede219b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:40.657396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661231e1-6726-4a6c-9941-d78ede219b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:40.668522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661231e1-6726-4a6c-9941-d78ede219b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:40.670946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661231e1-6726-4a6c-9941-d78ede219b64 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:40.725214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 928/1843] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:40.751655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30edb-a51d-4f5a-b7b0-a3090545dc52 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:40.753780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e0e30edb-a51d-4f5a-b7b0-a3090545dc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:40.784111 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30edb-a51d-4f5a-b7b0-a3090545dc52 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:40.795491 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:40.798193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:40.798889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:40.799470 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] PATH_INFO: `/v3/projects/502eac5866d043a4b56be279d1f13fc8` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:40.803734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=502eac5866d043a4b56be279d1f13fc8)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:40.806625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:40.892665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '502eac5866d043a4b56be279d1f13fc8', 'request_id': 'req-47cb3e93-0679-4726-adce-5214fe0bff57'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.901921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '502eac5866d043a4b56be279d1f13fc8', 'request_id': 'req-47cb3e93-0679-4726-adce-5214fe0bff57'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.908894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '502eac5866d043a4b56be279d1f13fc8', 'request_id': 'req-47cb3e93-0679-4726-adce-5214fe0bff57'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.937290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] The token cache is being invalidate because project 502eac5866d043a4b56be279d1f13fc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:40.937657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.938274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47cb3e93-0679-4726-adce-5214fe0bff57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:40.940090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 916/1844] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:40 2022] DELETE /identity/v3/projects/502eac5866d043a4b56be279d1f13fc8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:40.946747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6906c3-2301-4959-a9fb-a42efb29d4a5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:40.947033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6906c3-2301-4959-a9fb-a42efb29d4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:40.947369 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6906c3-2301-4959-a9fb-a42efb29d4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:40.966783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ef6906c3-2301-4959-a9fb-a42efb29d4a5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:40.968244 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ef6906c3-2301-4959-a9fb-a42efb29d4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:41.033031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 929/1845] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:41.042078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2b119-99af-4c39-b5d2-f8ac27364b65 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.042725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2b119-99af-4c39-b5d2-f8ac27364b65 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.042725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2b119-99af-4c39-b5d2-f8ac27364b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.066775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7fe2b119-99af-4c39-b5d2-f8ac27364b65 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:41.068040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7fe2b119-99af-4c39-b5d2-f8ac27364b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:41.076765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 917/1846] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:41.081859 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6906c3-2301-4959-a9fb-a42efb29d4a5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.091887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6906c3-2301-4959-a9fb-a42efb29d4a5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.096627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34054485-0e6b-4aec-a41a-c7a96921123f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.097318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34054485-0e6b-4aec-a41a-c7a96921123f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.097603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34054485-0e6b-4aec-a41a-c7a96921123f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.097883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34054485-0e6b-4aec-a41a-c7a96921123f admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.098524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34054485-0e6b-4aec-a41a-c7a96921123f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:43:41.098827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34054485-0e6b-4aec-a41a-c7a96921123f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.100420 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34054485-0e6b-4aec-a41a-c7a96921123f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.120465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 930/1847] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:43:41.124498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2b119-99af-4c39-b5d2-f8ac27364b65 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.134692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2b119-99af-4c39-b5d2-f8ac27364b65 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.139513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4f4af-0f80-4e48-9872-73f8579357ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.140126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4f4af-0f80-4e48-9872-73f8579357ff admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.140407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4f4af-0f80-4e48-9872-73f8579357ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.140720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4f4af-0f80-4e48-9872-73f8579357ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.141398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4f4af-0f80-4e48-9872-73f8579357ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.143523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4f4af-0f80-4e48-9872-73f8579357ff admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.177758 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 918/1848] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:41 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:43:41.181222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34054485-0e6b-4aec-a41a-c7a96921123f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.197955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34054485-0e6b-4aec-a41a-c7a96921123f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.218584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4f4af-0f80-4e48-9872-73f8579357ff admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.222855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f8e4f4af-0f80-4e48-9872-73f8579357ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:41.255239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:41.300926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbb2fb-62d3-4e76-8da8-423b26a356fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.301562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbb2fb-62d3-4e76-8da8-423b26a356fd admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.301844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbb2fb-62d3-4e76-8da8-423b26a356fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.302119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbb2fb-62d3-4e76-8da8-423b26a356fd admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.302764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbb2fb-62d3-4e76-8da8-423b26a356fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.305169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbb2fb-62d3-4e76-8da8-423b26a356fd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.314662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4f4af-0f80-4e48-9872-73f8579357ff admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.321066 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.321713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.322002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.322278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] PATH_INFO: `/v3/projects/d64df612870b46eb82b54cb7aa2c2adc` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.323526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64df612870b46eb82b54cb7aa2c2adc)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.325506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.331187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 931/1849] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:41 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:41.335636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbb2fb-62d3-4e76-8da8-423b26a356fd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.353847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbb2fb-62d3-4e76-8da8-423b26a356fd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.360168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825e90c-954a-41e2-8f29-480e930d157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.360828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e90c-954a-41e2-8f29-480e930d157a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.361132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e90c-954a-41e2-8f29-480e930d157a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.361409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e90c-954a-41e2-8f29-480e930d157a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.362311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825e90c-954a-41e2-8f29-480e930d157a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.363954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825e90c-954a-41e2-8f29-480e930d157a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.374916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 932/1850] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:41.379902 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64df612870b46eb82b54cb7aa2c2adc', 'request_id': 'req-0073c291-bced-4d52-b6d0-f35b2b8423fd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:41.380075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825e90c-954a-41e2-8f29-480e930d157a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.385594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64df612870b46eb82b54cb7aa2c2adc', 'request_id': 'req-0073c291-bced-4d52-b6d0-f35b2b8423fd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:41.389682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825e90c-954a-41e2-8f29-480e930d157a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.391827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64df612870b46eb82b54cb7aa2c2adc', 'request_id': 'req-0073c291-bced-4d52-b6d0-f35b2b8423fd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:41.399391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:41.407521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] The token cache is being invalidate because project d64df612870b46eb82b54cb7aa2c2adc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:41.407813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:41.408484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0073c291-bced-4d52-b6d0-f35b2b8423fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:41.410071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 919/1851] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:41 2022] DELETE /identity/v3/projects/d64df612870b46eb82b54cb7aa2c2adc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:41.414656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2fc51-fd72-4393-aeb8-06e6425f7763 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.414945 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2fc51-fd72-4393-aeb8-06e6425f7763 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.415237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2fc51-fd72-4393-aeb8-06e6425f7763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.434667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3af2fc51-fd72-4393-aeb8-06e6425f7763 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:41.435879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3af2fc51-fd72-4393-aeb8-06e6425f7763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:41.453187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dd0dc-1926-4adb-8403-e93a6be49c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.454002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dd0dc-1926-4adb-8403-e93a6be49c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.454285 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dd0dc-1926-4adb-8403-e93a6be49c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.454559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dd0dc-1926-4adb-8403-e93a6be49c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.455198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1dd0dc-1926-4adb-8403-e93a6be49c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.457400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1dd0dc-1926-4adb-8403-e93a6be49c06 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.467460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 933/1852] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:41.471362 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dd0dc-1926-4adb-8403-e93a6be49c06 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.473011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cb1dd0dc-1926-4adb-8403-e93a6be49c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:41.485000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dd0dc-1926-4adb-8403-e93a6be49c06 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.489720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73db73-8c32-4081-a8b7-41c51ee99818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.490389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73db73-8c32-4081-a8b7-41c51ee99818 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.490669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73db73-8c32-4081-a8b7-41c51ee99818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.490945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73db73-8c32-4081-a8b7-41c51ee99818 admin admin] PATH_INFO: `/v3/projects/57a4c07a0b8c4b7096f33df934d1b5b6/users/b3d8e3575bab47beaef2ed34da65373a/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.501101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 920/1853] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:41.507031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73db73-8c32-4081-a8b7-41c51ee99818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a4c07a0b8c4b7096f33df934d1b5b6, user_id=b3d8e3575bab47beaef2ed34da65373a, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.509769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73db73-8c32-4081-a8b7-41c51ee99818 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.510677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76728d5e-e268-445e-9e83-2d8afc16da3f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.510976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76728d5e-e268-445e-9e83-2d8afc16da3f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.511290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76728d5e-e268-445e-9e83-2d8afc16da3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.525347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 934/1854] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:41 2022] PUT /identity/v3/projects/57a4c07a0b8c4b7096f33df934d1b5b6/users/b3d8e3575bab47beaef2ed34da65373a/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:41.530009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa276e1-3770-4aa0-b351-cd53fd9183ef None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.530303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa276e1-3770-4aa0-b351-cd53fd9183ef None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.530582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa276e1-3770-4aa0-b351-cd53fd9183ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.533301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-76728d5e-e268-445e-9e83-2d8afc16da3f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:41.534653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-76728d5e-e268-445e-9e83-2d8afc16da3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:41.550004 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7aa276e1-3770-4aa0-b351-cd53fd9183ef None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:41.551331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7aa276e1-3770-4aa0-b351-cd53fd9183ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:41.579723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:41.602905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 921/1855] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:41.607371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76728d5e-e268-445e-9e83-2d8afc16da3f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.615667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 935/1856] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:41.622339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76728d5e-e268-445e-9e83-2d8afc16da3f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.627051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa276e1-3770-4aa0-b351-cd53fd9183ef None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.628906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc4b3d-ee1d-42da-9eb6-ead4b41fa884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.629192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7aa276e1-3770-4aa0-b351-cd53fd9183ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:41.629755 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc4b3d-ee1d-42da-9eb6-ead4b41fa884 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.630047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc4b3d-ee1d-42da-9eb6-ead4b41fa884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.630391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc4b3d-ee1d-42da-9eb6-ead4b41fa884 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.631025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc4b3d-ee1d-42da-9eb6-ead4b41fa884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:43:41.631328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc4b3d-ee1d-42da-9eb6-ead4b41fa884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.633009 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc4b3d-ee1d-42da-9eb6-ead4b41fa884 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.647190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 922/1857] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:43:41.651629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc4b3d-ee1d-42da-9eb6-ead4b41fa884 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.662869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc4b3d-ee1d-42da-9eb6-ead4b41fa884 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.667847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479628f4-5638-4884-89fe-d55b0c2a2ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.668471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479628f4-5638-4884-89fe-d55b0c2a2ead admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.669239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479628f4-5638-4884-89fe-d55b0c2a2ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.669521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479628f4-5638-4884-89fe-d55b0c2a2ead admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.670185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479628f4-5638-4884-89fe-d55b0c2a2ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.672085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479628f4-5638-4884-89fe-d55b0c2a2ead admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.697732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa276e1-3770-4aa0-b351-cd53fd9183ef None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.701875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357725a-c371-4d61-b0f5-30d885984af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.702489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357725a-c371-4d61-b0f5-30d885984af0 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.702770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357725a-c371-4d61-b0f5-30d885984af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.703039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357725a-c371-4d61-b0f5-30d885984af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.708667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 923/1858] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:41 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:43:41.711412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479628f4-5638-4884-89fe-d55b0c2a2ead admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.713632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357725a-c371-4d61-b0f5-30d885984af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.715062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357725a-c371-4d61-b0f5-30d885984af0 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.726406 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 936/1859] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:41.815088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479628f4-5638-4884-89fe-d55b0c2a2ead admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.820971 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a17ec8-2af9-4a48-94f6-ca904db09374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.822580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a17ec8-2af9-4a48-94f6-ca904db09374 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.822580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a17ec8-2af9-4a48-94f6-ca904db09374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.822580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a17ec8-2af9-4a48-94f6-ca904db09374 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.823489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a17ec8-2af9-4a48-94f6-ca904db09374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.825947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a17ec8-2af9-4a48-94f6-ca904db09374 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.855128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 924/1860] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:41 2022] POST /identity/v3/users => generated 286 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:43:41.856576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357725a-c371-4d61-b0f5-30d885984af0 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.873135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357725a-c371-4d61-b0f5-30d885984af0 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.878899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa86031-726c-43b7-9b6c-6555ac9fe8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.879562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa86031-726c-43b7-9b6c-6555ac9fe8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.879790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa86031-726c-43b7-9b6c-6555ac9fe8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.880078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa86031-726c-43b7-9b6c-6555ac9fe8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.881068 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa86031-726c-43b7-9b6c-6555ac9fe8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.882329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa86031-726c-43b7-9b6c-6555ac9fe8d0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.894037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 937/1861] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:43:41.899112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a17ec8-2af9-4a48-94f6-ca904db09374 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.911477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a17ec8-2af9-4a48-94f6-ca904db09374 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.917347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de381fc1-67a5-439c-851e-71dbc4b9b2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.918539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de381fc1-67a5-439c-851e-71dbc4b9b2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.918876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de381fc1-67a5-439c-851e-71dbc4b9b2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.919155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de381fc1-67a5-439c-851e-71dbc4b9b2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.919955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de381fc1-67a5-439c-851e-71dbc4b9b2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.921878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de381fc1-67a5-439c-851e-71dbc4b9b2b3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.935004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 925/1862] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:41.938774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa86031-726c-43b7-9b6c-6555ac9fe8d0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:41.952572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa86031-726c-43b7-9b6c-6555ac9fe8d0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:41.958498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23377ef1-5704-4562-8924-34776b3f98e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:41.959268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377ef1-5704-4562-8924-34776b3f98e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:41.959752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377ef1-5704-4562-8924-34776b3f98e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:41.960119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377ef1-5704-4562-8924-34776b3f98e2 admin admin] PATH_INFO: `/v3/projects/d031a00a30444b26ab01aef72f69287f/users/0b83cf11d81e4e148009b71639c72128/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:41.981078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23377ef1-5704-4562-8924-34776b3f98e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d031a00a30444b26ab01aef72f69287f, user_id=0b83cf11d81e4e148009b71639c72128, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:41.984064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23377ef1-5704-4562-8924-34776b3f98e2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:41.988615 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:42.018886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ed925-b948-4e9b-8150-1c4da2c49877 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:42.019461 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ed925-b948-4e9b-8150-1c4da2c49877 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:42.019590 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ed925-b948-4e9b-8150-1c4da2c49877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:42.020973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 938/1863] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:41 2022] PUT /identity/v3/projects/d031a00a30444b26ab01aef72f69287f/users/0b83cf11d81e4e148009b71639c72128/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:42.086888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-bc7ed925-b948-4e9b-8150-1c4da2c49877 None None] MFA Rules not processed for user `0b83cf11d81e4e148009b71639c72128`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:42.089519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bc7ed925-b948-4e9b-8150-1c4da2c49877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:42.184640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 926/1864] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:43:42.197455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23377ef1-5704-4562-8924-34776b3f98e2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:42.199796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-23377ef1-5704-4562-8924-34776b3f98e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:42.281471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23377ef1-5704-4562-8924-34776b3f98e2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:42.287278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b05bfe-7425-4eff-a370-e021a2b485ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:42.288074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b05bfe-7425-4eff-a370-e021a2b485ff service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:42.288450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b05bfe-7425-4eff-a370-e021a2b485ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:42.289040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b05bfe-7425-4eff-a370-e021a2b485ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:42.302113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b05bfe-7425-4eff-a370-e021a2b485ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:42.305221 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b05bfe-7425-4eff-a370-e021a2b485ff service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:42.322366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 939/1865] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:44.269257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f707dfb-dc69-41d0-baae-f850168a9a3e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:44.269966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f707dfb-dc69-41d0-baae-f850168a9a3e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:44.270202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f707dfb-dc69-41d0-baae-f850168a9a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:44.331445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5f707dfb-dc69-41d0-baae-f850168a9a3e None None] MFA Rules not processed for user `b3d8e3575bab47beaef2ed34da65373a`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:44.333174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5f707dfb-dc69-41d0-baae-f850168a9a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:44.343353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:44.433582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f31e11-9976-4879-b518-d673433e5401 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:44.434039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f31e11-9976-4879-b518-d673433e5401 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:44.434039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f31e11-9976-4879-b518-d673433e5401 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:44.436027 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 940/1866] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:43:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:43:44.459636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 927/1867] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:44.464116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f31e11-9976-4879-b518-d673433e5401 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:44.470399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36181c71-4c70-4977-9e62-f4860954330c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:44.470977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36181c71-4c70-4977-9e62-f4860954330c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:44.474237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36181c71-4c70-4977-9e62-f4860954330c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:44.529529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-36181c71-4c70-4977-9e62-f4860954330c None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:44.535493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-36181c71-4c70-4977-9e62-f4860954330c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:44.630023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f31e11-9976-4879-b518-d673433e5401 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:44.644187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1a3a2-3a9c-4860-8718-35a915c8f935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:44.644798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1a3a2-3a9c-4860-8718-35a915c8f935 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:44.645074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1a3a2-3a9c-4860-8718-35a915c8f935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:44.645343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1a3a2-3a9c-4860-8718-35a915c8f935 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:44.645982 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1a3a2-3a9c-4860-8718-35a915c8f935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:44.647881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1a3a2-3a9c-4860-8718-35a915c8f935 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:44.694689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 941/1868] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:43:44 2022] POST /identity/v3/projects => generated 371 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:43:44.698904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1a3a2-3a9c-4860-8718-35a915c8f935 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:44.719364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 928/1869] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:43:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:44.781952 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1a3a2-3a9c-4860-8718-35a915c8f935 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:44.789461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37f1bd-6418-4e16-9d6e-89a298192b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:44.790115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37f1bd-6418-4e16-9d6e-89a298192b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:44.790397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37f1bd-6418-4e16-9d6e-89a298192b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:44.790670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37f1bd-6418-4e16-9d6e-89a298192b7e admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:44.791311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37f1bd-6418-4e16-9d6e-89a298192b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:44.793287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37f1bd-6418-4e16-9d6e-89a298192b7e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:44.823147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 942/1870] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:43:44 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:43:44.824402 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36181c71-4c70-4977-9e62-f4860954330c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:44.836621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36181c71-4c70-4977-9e62-f4860954330c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:44.841311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e24ba5-c705-414b-993b-3507d2939ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:44.841864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24ba5-c705-414b-993b-3507d2939ead admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:44.842149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24ba5-c705-414b-993b-3507d2939ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:44.842427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24ba5-c705-414b-993b-3507d2939ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:44.843082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e24ba5-c705-414b-993b-3507d2939ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:44.844752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e24ba5-c705-414b-993b-3507d2939ead admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:44.856567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 929/1871] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:43:44.860985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37f1bd-6418-4e16-9d6e-89a298192b7e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:44.871620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37f1bd-6418-4e16-9d6e-89a298192b7e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:44.876284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d09255-da6b-4d2c-88f6-37f8ed96929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:44.876975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d09255-da6b-4d2c-88f6-37f8ed96929e admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:44.877259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d09255-da6b-4d2c-88f6-37f8ed96929e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:44.877566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d09255-da6b-4d2c-88f6-37f8ed96929e admin admin] PATH_INFO: `/v3/projects/2f61edd63e4941079b8fe1d29c8736b6/users/53ae977efe3143e5bbae8b0c738e52b7/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:44.888035 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:44.900553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d09255-da6b-4d2c-88f6-37f8ed96929e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f61edd63e4941079b8fe1d29c8736b6, user_id=53ae977efe3143e5bbae8b0c738e52b7, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:44.903327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d09255-da6b-4d2c-88f6-37f8ed96929e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:44.927272 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 943/1872] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:44 2022] PUT /identity/v3/projects/2f61edd63e4941079b8fe1d29c8736b6/users/53ae977efe3143e5bbae8b0c738e52b7/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:44.929247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e24ba5-c705-414b-993b-3507d2939ead admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:44.944687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e24ba5-c705-414b-993b-3507d2939ead admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:45.007073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d09255-da6b-4d2c-88f6-37f8ed96929e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:45.010402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-59d09255-da6b-4d2c-88f6-37f8ed96929e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:45.023467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5d804-5bc6-42cf-bc49-1e268556a297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:45.024179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5d804-5bc6-42cf-bc49-1e268556a297 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:45.024572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5d804-5bc6-42cf-bc49-1e268556a297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:45.025022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5d804-5bc6-42cf-bc49-1e268556a297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:45.026387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5d804-5bc6-42cf-bc49-1e268556a297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:45.028146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5d804-5bc6-42cf-bc49-1e268556a297 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:45.040256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 930/1873] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:43:45.045859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5d804-5bc6-42cf-bc49-1e268556a297 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:45.078368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5d804-5bc6-42cf-bc49-1e268556a297 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:45.103882 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf77bb8-f0b7-48d9-91e8-a846222f1a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:45.105354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf77bb8-f0b7-48d9-91e8-a846222f1a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:45.105965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf77bb8-f0b7-48d9-91e8-a846222f1a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:45.106319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf77bb8-f0b7-48d9-91e8-a846222f1a49 admin admin] PATH_INFO: `/v3/projects/2f61edd63e4941079b8fe1d29c8736b6/users/53ae977efe3143e5bbae8b0c738e52b7/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:45.109629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf77bb8-f0b7-48d9-91e8-a846222f1a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f61edd63e4941079b8fe1d29c8736b6, user_id=53ae977efe3143e5bbae8b0c738e52b7, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:45.119727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf77bb8-f0b7-48d9-91e8-a846222f1a49 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:45.136539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d09255-da6b-4d2c-88f6-37f8ed96929e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:45.140529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 931/1874] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:43:45 2022] PUT /identity/v3/projects/2f61edd63e4941079b8fe1d29c8736b6/users/53ae977efe3143e5bbae8b0c738e52b7/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:43:45.207667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b961dd7-4f8a-48c7-a1ed-bd79dde50f22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:45.208314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b961dd7-4f8a-48c7-a1ed-bd79dde50f22 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:45.209254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b961dd7-4f8a-48c7-a1ed-bd79dde50f22 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:45.209533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b961dd7-4f8a-48c7-a1ed-bd79dde50f22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:45.220312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b961dd7-4f8a-48c7-a1ed-bd79dde50f22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:45.223845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b961dd7-4f8a-48c7-a1ed-bd79dde50f22 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:45.235612 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 944/1875] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:43:46.179434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11c2a9-555c-4455-a0b4-726bdbc8ed27 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:46.180323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11c2a9-555c-4455-a0b4-726bdbc8ed27 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:46.180865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11c2a9-555c-4455-a0b4-726bdbc8ed27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:46.213181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4b11c2a9-555c-4455-a0b4-726bdbc8ed27 None None] MFA Rules not processed for user `53ae977efe3143e5bbae8b0c738e52b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:46.214572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4b11c2a9-555c-4455-a0b4-726bdbc8ed27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:46.219703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:46.319119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 932/1876] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:43:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:46.333775 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b961dd7-4f8a-48c7-a1ed-bd79dde50f22 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:46.411880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b961dd7-4f8a-48c7-a1ed-bd79dde50f22 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:46.416857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0082a3c-1038-4fb2-a51c-f4b53225bd78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:46.417484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0082a3c-1038-4fb2-a51c-f4b53225bd78 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:46.417765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0082a3c-1038-4fb2-a51c-f4b53225bd78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:46.418038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0082a3c-1038-4fb2-a51c-f4b53225bd78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:46.452760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:46.495961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0082a3c-1038-4fb2-a51c-f4b53225bd78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:46.497444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0082a3c-1038-4fb2-a51c-f4b53225bd78 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:46.507458 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 945/1877] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:54.258460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11c2a9-555c-4455-a0b4-726bdbc8ed27 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:54.268454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11c2a9-555c-4455-a0b4-726bdbc8ed27 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:54.272196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f62d4f-69f2-473a-89b9-201c5fff3816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:54.272836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f62d4f-69f2-473a-89b9-201c5fff3816 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:54.273121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f62d4f-69f2-473a-89b9-201c5fff3816 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:54.273394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f62d4f-69f2-473a-89b9-201c5fff3816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:54.283872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f62d4f-69f2-473a-89b9-201c5fff3816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:54.284289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f62d4f-69f2-473a-89b9-201c5fff3816 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:54.302699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 933/1878] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:56.502201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=16 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:04.295977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=33 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:08.018319 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ce76f-82b8-467c-a4cd-636b2eaf6172 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:08.018918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ce76f-82b8-467c-a4cd-636b2eaf6172 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:08.019045 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ce76f-82b8-467c-a4cd-636b2eaf6172 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:08.022004 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 946/1879] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:44:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:44:08.029826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72ab74-7288-44e3-a202-578635447df2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:08.030383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72ab74-7288-44e3-a202-578635447df2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:08.030675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72ab74-7288-44e3-a202-578635447df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:08.080114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6b72ab74-7288-44e3-a202-578635447df2 None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:08.083667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6b72ab74-7288-44e3-a202-578635447df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:08.103137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 934/1880] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:44:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:08.116645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ce76f-82b8-467c-a4cd-636b2eaf6172 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:08.118599 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b51ce76f-82b8-467c-a4cd-636b2eaf6172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:08.139255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ce76f-82b8-467c-a4cd-636b2eaf6172 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:08.146692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340af62-f531-4e42-9bc4-20c43b9df933 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:08.147307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340af62-f531-4e42-9bc4-20c43b9df933 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:08.147592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340af62-f531-4e42-9bc4-20c43b9df933 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:08.147866 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340af62-f531-4e42-9bc4-20c43b9df933 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:08.157168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340af62-f531-4e42-9bc4-20c43b9df933 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:08.158775 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340af62-f531-4e42-9bc4-20c43b9df933 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:08.170402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 947/1881] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:18.054257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:18.166123 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:19.161163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72ab74-7288-44e3-a202-578635447df2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:19.249778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72ab74-7288-44e3-a202-578635447df2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:19.256094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:19.257012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:19.257450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:19.257939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] PATH_INFO: `/v3/users/0b83cf11d81e4e148009b71639c72128` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:19.259653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b83cf11d81e4e148009b71639c72128)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:19.261740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:19.321883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b83cf11d81e4e148009b71639c72128'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:19.322605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b83cf11d81e4e148009b71639c72128'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:19.333080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b83cf11d81e4e148009b71639c72128'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:19.342642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b83cf11d81e4e148009b71639c72128'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:19.343390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b83cf11d81e4e148009b71639c72128'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:19.367032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 935/1882] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:44:19 2022] DELETE /identity/v3/users/0b83cf11d81e4e148009b71639c72128 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:19.371826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675e26d-e775-473c-af65-49ee656c17ed None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:19.372377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675e26d-e775-473c-af65-49ee656c17ed None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:19.372377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675e26d-e775-473c-af65-49ee656c17ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:19.394160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8675e26d-e775-473c-af65-49ee656c17ed None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:19.395392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8675e26d-e775-473c-af65-49ee656c17ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:19.469662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 948/1883] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:19.479259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:19.480763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:19.555303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0eec2-84e2-4ff0-ac92-b11064898eac admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:19.560338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8ca6a-7f8b-4236-b436-f8f5d03e078a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:19.560338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ca6a-7f8b-4236-b436-f8f5d03e078a service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:19.560338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ca6a-7f8b-4236-b436-f8f5d03e078a service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:19.560338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ca6a-7f8b-4236-b436-f8f5d03e078a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:19.571706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8ca6a-7f8b-4236-b436-f8f5d03e078a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:19.573184 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8ca6a-7f8b-4236-b436-f8f5d03e078a service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:19.584475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 936/1884] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:19.905000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675e26d-e775-473c-af65-49ee656c17ed None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:19.906739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8675e26d-e775-473c-af65-49ee656c17ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:19.926703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675e26d-e775-473c-af65-49ee656c17ed None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:19.941763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:19.943139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:19.943623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:19.944075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] PATH_INFO: `/v3/projects/d031a00a30444b26ab01aef72f69287f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:19.945878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d031a00a30444b26ab01aef72f69287f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:19.948666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:20.023578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd031a00a30444b26ab01aef72f69287f', 'request_id': 'req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:20.038290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd031a00a30444b26ab01aef72f69287f', 'request_id': 'req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:20.048317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd031a00a30444b26ab01aef72f69287f', 'request_id': 'req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:20.074184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] The token cache is being invalidate because project d031a00a30444b26ab01aef72f69287f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:44:20.074639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:20.075448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbe00ef3-c44f-4dd6-ba99-c31211f5aa6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:20.078155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 949/1885] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:44:19 2022] DELETE /identity/v3/projects/d031a00a30444b26ab01aef72f69287f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:20.083307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e203f21-4060-40dd-b5bd-04355c66e013 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:20.083605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e203f21-4060-40dd-b5bd-04355c66e013 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:20.083892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e203f21-4060-40dd-b5bd-04355c66e013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:20.139000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3e203f21-4060-40dd-b5bd-04355c66e013 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:20.140200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3e203f21-4060-40dd-b5bd-04355c66e013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:20.171428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:20.237526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 937/1886] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:44:20.245223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0dece-6216-43c9-bb27-8c466853c622 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:20.245610 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0dece-6216-43c9-bb27-8c466853c622 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:20.245847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0dece-6216-43c9-bb27-8c466853c622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:20.269487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-83c0dece-6216-43c9-bb27-8c466853c622 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:20.272121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-83c0dece-6216-43c9-bb27-8c466853c622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:20.282796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 950/1887] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:20.287468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e203f21-4060-40dd-b5bd-04355c66e013 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:20.298573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e203f21-4060-40dd-b5bd-04355c66e013 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:20.306042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d2ceb-38ba-4307-871c-d20306970a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:20.306660 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d2ceb-38ba-4307-871c-d20306970a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:20.307116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d2ceb-38ba-4307-871c-d20306970a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:20.307547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d2ceb-38ba-4307-871c-d20306970a57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:20.308556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d2ceb-38ba-4307-871c-d20306970a57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:44:20.309526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d2ceb-38ba-4307-871c-d20306970a57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:20.311318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d2ceb-38ba-4307-871c-d20306970a57 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:20.327934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 938/1888] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:44:20.332813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0dece-6216-43c9-bb27-8c466853c622 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:20.344773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0dece-6216-43c9-bb27-8c466853c622 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:20.350881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7aceb-4526-402d-bc92-f528ff09b34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:20.351495 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7aceb-4526-402d-bc92-f528ff09b34a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:20.351772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7aceb-4526-402d-bc92-f528ff09b34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:20.352250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7aceb-4526-402d-bc92-f528ff09b34a admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:20.352928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7aceb-4526-402d-bc92-f528ff09b34a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:20.354865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7aceb-4526-402d-bc92-f528ff09b34a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:20.386377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 951/1889] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:44:20 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:20.387513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d2ceb-38ba-4307-871c-d20306970a57 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:20.397979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d2ceb-38ba-4307-871c-d20306970a57 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:20.462690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6fb96-9139-4527-9c88-3d239fd62743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:20.463323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6fb96-9139-4527-9c88-3d239fd62743 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:20.463596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6fb96-9139-4527-9c88-3d239fd62743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:20.463873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6fb96-9139-4527-9c88-3d239fd62743 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:20.464550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6fb96-9139-4527-9c88-3d239fd62743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:20.466730 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6fb96-9139-4527-9c88-3d239fd62743 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:20.490568 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 939/1890] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:44:20 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:20.494602 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7aceb-4526-402d-bc92-f528ff09b34a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:20.505470 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7aceb-4526-402d-bc92-f528ff09b34a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:20.510329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2b065-d02f-4b6b-b07c-fb0c6941636c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:20.510936 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2b065-d02f-4b6b-b07c-fb0c6941636c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:20.511220 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2b065-d02f-4b6b-b07c-fb0c6941636c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:20.511494 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2b065-d02f-4b6b-b07c-fb0c6941636c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:20.512135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2b065-d02f-4b6b-b07c-fb0c6941636c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:20.513774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2b065-d02f-4b6b-b07c-fb0c6941636c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:20.527226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 952/1891] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:20.528577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6fb96-9139-4527-9c88-3d239fd62743 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:20.539632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6fb96-9139-4527-9c88-3d239fd62743 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:20.544695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c7403-c0e5-47eb-a496-0d8df47fd5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:20.545309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c7403-c0e5-47eb-a496-0d8df47fd5ab admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:20.545601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c7403-c0e5-47eb-a496-0d8df47fd5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:20.545882 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c7403-c0e5-47eb-a496-0d8df47fd5ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:20.546534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c7403-c0e5-47eb-a496-0d8df47fd5ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:20.548147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c7403-c0e5-47eb-a496-0d8df47fd5ab admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:20.562068 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 940/1892] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:20.567082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2b065-d02f-4b6b-b07c-fb0c6941636c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:20.577353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2b065-d02f-4b6b-b07c-fb0c6941636c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:20.582656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa44f1-ec84-4e99-9f93-56326f0210e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:20.583317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa44f1-ec84-4e99-9f93-56326f0210e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:20.583598 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa44f1-ec84-4e99-9f93-56326f0210e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:20.583875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa44f1-ec84-4e99-9f93-56326f0210e0 admin admin] PATH_INFO: `/v3/projects/3b4f340d2f194b33bf5bde41e61fb1a5/users/14c2fa3118c74e128f3307ab1efcb0b4/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:20.599515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa44f1-ec84-4e99-9f93-56326f0210e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4f340d2f194b33bf5bde41e61fb1a5, user_id=14c2fa3118c74e128f3307ab1efcb0b4, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:20.602231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa44f1-ec84-4e99-9f93-56326f0210e0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:20.614754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 953/1893] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:20 2022] PUT /identity/v3/projects/3b4f340d2f194b33bf5bde41e61fb1a5/users/14c2fa3118c74e128f3307ab1efcb0b4/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:20.625225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4de43f-89e7-4ea8-979c-89f143b95421 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:20.625519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4de43f-89e7-4ea8-979c-89f143b95421 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:20.625831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4de43f-89e7-4ea8-979c-89f143b95421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:20.654246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ae4de43f-89e7-4ea8-979c-89f143b95421 None None] MFA Rules not processed for user `14c2fa3118c74e128f3307ab1efcb0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:20.655478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ae4de43f-89e7-4ea8-979c-89f143b95421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:20.721664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 941/1894] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:20.733173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa44f1-ec84-4e99-9f93-56326f0210e0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:20.734818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-42fa44f1-ec84-4e99-9f93-56326f0210e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:20.758450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:20.814172 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa44f1-ec84-4e99-9f93-56326f0210e0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:20.819123 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b0379-0cad-4b36-b86f-cedab0091a02 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ec048788904bc58245e878a8644091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:20.819726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b0379-0cad-4b36-b86f-cedab0091a02 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:20.820008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b0379-0cad-4b36-b86f-cedab0091a02 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:20.820280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b0379-0cad-4b36-b86f-cedab0091a02 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:20.829257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b0379-0cad-4b36-b86f-cedab0091a02 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:20.831638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b0379-0cad-4b36-b86f-cedab0091a02 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:20.840441 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 954/1895] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:25.379273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65966c-dd73-44eb-9b83-3546b1e22cfa None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:25.380010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65966c-dd73-44eb-9b83-3546b1e22cfa None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:25.380010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65966c-dd73-44eb-9b83-3546b1e22cfa None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:25.381592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 942/1896] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:44:25.413443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919e828-60f6-4e2d-8a3b-18e94d3e5808 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:25.413997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919e828-60f6-4e2d-8a3b-18e94d3e5808 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:25.413997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919e828-60f6-4e2d-8a3b-18e94d3e5808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:25.451492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5919e828-60f6-4e2d-8a3b-18e94d3e5808 None None] MFA Rules not processed for user `f9114a71a3a247188b263d87b6cb12c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:25.454394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5919e828-60f6-4e2d-8a3b-18e94d3e5808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:25.571283 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 955/1897] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:44:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:25.583752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65966c-dd73-44eb-9b83-3546b1e22cfa None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:25.587597 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9a65966c-dd73-44eb-9b83-3546b1e22cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:25.605616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65966c-dd73-44eb-9b83-3546b1e22cfa None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:25.612780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07504a-d033-471b-ac15-708fef2ddc44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:25.613407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07504a-d033-471b-ac15-708fef2ddc44 service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:25.613692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07504a-d033-471b-ac15-708fef2ddc44 service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:25.613970 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07504a-d033-471b-ac15-708fef2ddc44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:25.624850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07504a-d033-471b-ac15-708fef2ddc44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:25.626932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07504a-d033-471b-ac15-708fef2ddc44 service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:25.639428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 943/1898] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:26.371843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919e828-60f6-4e2d-8a3b-18e94d3e5808 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:26.373278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5919e828-60f6-4e2d-8a3b-18e94d3e5808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:26.462395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919e828-60f6-4e2d-8a3b-18e94d3e5808 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:26.469205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:26.469837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:26.470114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:26.470382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] PATH_INFO: `/v3/users/b3d8e3575bab47beaef2ed34da65373a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:26.471566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d8e3575bab47beaef2ed34da65373a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:26.473519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:26.530907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:26.542400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d8e3575bab47beaef2ed34da65373a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:26.553673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d8e3575bab47beaef2ed34da65373a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:26.554618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d8e3575bab47beaef2ed34da65373a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:26.555361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d8e3575bab47beaef2ed34da65373a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:26.573535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d8e3575bab47beaef2ed34da65373a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:26.593787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 956/1899] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:44:26 2022] DELETE /identity/v3/users/b3d8e3575bab47beaef2ed34da65373a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:26.792084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07504a-d033-471b-ac15-708fef2ddc44 service placement] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:26.806550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:26.811846 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07504a-d033-471b-ac15-708fef2ddc44 service placement] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:26.882586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:26.883214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:26.883491 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:26.883760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] PATH_INFO: `/v3/projects/57a4c07a0b8c4b7096f33df934d1b5b6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:26.884907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a4c07a0b8c4b7096f33df934d1b5b6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:26.886800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:26.937001 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a4c07a0b8c4b7096f33df934d1b5b6', 'request_id': 'req-275ba0fd-1a5a-41bb-9327-2e2e49464823'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:26.943309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a4c07a0b8c4b7096f33df934d1b5b6', 'request_id': 'req-275ba0fd-1a5a-41bb-9327-2e2e49464823'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:26.949843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a4c07a0b8c4b7096f33df934d1b5b6', 'request_id': 'req-275ba0fd-1a5a-41bb-9327-2e2e49464823'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:26.963212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] The token cache is being invalidate because project 57a4c07a0b8c4b7096f33df934d1b5b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:44:26.963498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:26.964145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:26.965858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 944/1900] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:44:26 2022] DELETE /identity/v3/projects/57a4c07a0b8c4b7096f33df934d1b5b6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:26.970009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:26.973224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:26.991455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ad5c2-face-4603-a267-ab347d65f7a8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:27.051995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:27.052851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:27.053146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:27.053418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] PATH_INFO: `/v3/users/53ae977efe3143e5bbae8b0c738e52b7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:27.054720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53ae977efe3143e5bbae8b0c738e52b7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:27.057075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:27.113622 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53ae977efe3143e5bbae8b0c738e52b7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.124638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53ae977efe3143e5bbae8b0c738e52b7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.125427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53ae977efe3143e5bbae8b0c738e52b7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.126033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53ae977efe3143e5bbae8b0c738e52b7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.134978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53ae977efe3143e5bbae8b0c738e52b7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.156897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 957/1901] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:44:26 2022] DELETE /identity/v3/users/53ae977efe3143e5bbae8b0c738e52b7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:44:27.329809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:27.332530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:27.349317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ba0fd-1a5a-41bb-9327-2e2e49464823 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:27.373129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:27.374929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:27.438988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a57d7b-4043-4ca2-990f-6e02855ee933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:27.440868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a57d7b-4043-4ca2-990f-6e02855ee933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:27.441801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a57d7b-4043-4ca2-990f-6e02855ee933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:27.442745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a57d7b-4043-4ca2-990f-6e02855ee933 admin admin] PATH_INFO: `/v3/users/41d433ebf9f642aabae47f99a0673faa` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:27.444553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a57d7b-4043-4ca2-990f-6e02855ee933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d433ebf9f642aabae47f99a0673faa)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:27.448127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a57d7b-4043-4ca2-990f-6e02855ee933 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:27.468152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:27.504086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2f327-fb1c-4051-b189-0b1caba26446 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:27.509624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:27.510340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:27.510676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:27.510891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] PATH_INFO: `/v3/projects/2f61edd63e4941079b8fe1d29c8736b6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:27.512066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f61edd63e4941079b8fe1d29c8736b6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:27.514246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:27.522647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:27.532363 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-06a57d7b-4043-4ca2-990f-6e02855ee933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d433ebf9f642aabae47f99a0673faa'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.533877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-06a57d7b-4043-4ca2-990f-6e02855ee933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d433ebf9f642aabae47f99a0673faa'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.548319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-06a57d7b-4043-4ca2-990f-6e02855ee933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d433ebf9f642aabae47f99a0673faa'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.559757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-06a57d7b-4043-4ca2-990f-6e02855ee933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d433ebf9f642aabae47f99a0673faa'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.560570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-06a57d7b-4043-4ca2-990f-6e02855ee933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d433ebf9f642aabae47f99a0673faa'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.591807 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 945/1902] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:44:27 2022] DELETE /identity/v3/users/41d433ebf9f642aabae47f99a0673faa => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:27.597777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fad8d4-c049-471e-ac57-a6a724b74bf0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:27.598283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fad8d4-c049-471e-ac57-a6a724b74bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:27.598713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fad8d4-c049-471e-ac57-a6a724b74bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:27.617545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f61edd63e4941079b8fe1d29c8736b6', 'request_id': 'req-d2bed43d-2dad-4651-a348-bed74448584b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.630143 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f61edd63e4941079b8fe1d29c8736b6', 'request_id': 'req-d2bed43d-2dad-4651-a348-bed74448584b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.648387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f61edd63e4941079b8fe1d29c8736b6', 'request_id': 'req-d2bed43d-2dad-4651-a348-bed74448584b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.649831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c1fad8d4-c049-471e-ac57-a6a724b74bf0 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:27.662449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c1fad8d4-c049-471e-ac57-a6a724b74bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:27.682509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] The token cache is being invalidate because project 2f61edd63e4941079b8fe1d29c8736b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:44:27.682509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.682509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d2bed43d-2dad-4651-a348-bed74448584b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:27.682509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 958/1903] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:44:27 2022] DELETE /identity/v3/projects/2f61edd63e4941079b8fe1d29c8736b6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:27.687392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b84305-6c14-44b0-befa-aea20d5cd683 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:27.687700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b84305-6c14-44b0-befa-aea20d5cd683 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:27.687964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b84305-6c14-44b0-befa-aea20d5cd683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:27.756156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-88b84305-6c14-44b0-befa-aea20d5cd683 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:27.758640 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-88b84305-6c14-44b0-befa-aea20d5cd683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:27.890471 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 946/1904] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:27.962181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fad8d4-c049-471e-ac57-a6a724b74bf0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:27.965222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c1fad8d4-c049-471e-ac57-a6a724b74bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:27.995057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 959/1905] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:28.004175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbb134-c4c4-4ce4-bac7-924327ca87e2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:28.004870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbb134-c4c4-4ce4-bac7-924327ca87e2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:28.007182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbb134-c4c4-4ce4-bac7-924327ca87e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:28.072959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5ccbb134-c4c4-4ce4-bac7-924327ca87e2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:28.074178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5ccbb134-c4c4-4ce4-bac7-924327ca87e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:28.119988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 960/1906] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:28.124254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbb134-c4c4-4ce4-bac7-924327ca87e2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:28.144130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbb134-c4c4-4ce4-bac7-924327ca87e2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:28.152668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fad8d4-c049-471e-ac57-a6a724b74bf0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:28.153511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785075d2-8336-4d93-9d57-d9057ffa94f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:28.154136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785075d2-8336-4d93-9d57-d9057ffa94f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:28.154408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785075d2-8336-4d93-9d57-d9057ffa94f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:28.154680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785075d2-8336-4d93-9d57-d9057ffa94f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:28.155330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785075d2-8336-4d93-9d57-d9057ffa94f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:44:28.155635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785075d2-8336-4d93-9d57-d9057ffa94f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:28.158834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785075d2-8336-4d93-9d57-d9057ffa94f0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:28.165334 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5964aea-843f-4421-86cc-6a3e5d3edfd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:28.168984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5964aea-843f-4421-86cc-6a3e5d3edfd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:28.170134 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5964aea-843f-4421-86cc-6a3e5d3edfd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:28.170711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5964aea-843f-4421-86cc-6a3e5d3edfd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:28.192524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 961/1907] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:44:28.195301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785075d2-8336-4d93-9d57-d9057ffa94f0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:28.196381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5964aea-843f-4421-86cc-6a3e5d3edfd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:28.200811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5964aea-843f-4421-86cc-6a3e5d3edfd9 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:28.217070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785075d2-8336-4d93-9d57-d9057ffa94f0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:28.226832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd10dd-1554-44c4-9f3d-7839598731dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:28.227701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd10dd-1554-44c4-9f3d-7839598731dd admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:28.229981 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd10dd-1554-44c4-9f3d-7839598731dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:28.230506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd10dd-1554-44c4-9f3d-7839598731dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:28.231545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd10dd-1554-44c4-9f3d-7839598731dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:28.233916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd10dd-1554-44c4-9f3d-7839598731dd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:28.250675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 947/1908] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:44:28.301864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5964aea-843f-4421-86cc-6a3e5d3edfd9 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:28.306512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 962/1909] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:44:28 2022] POST /identity/v3/projects => generated 361 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:28.316104 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5964aea-843f-4421-86cc-6a3e5d3edfd9 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:28.418505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:28.430396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23af236-5f24-4660-9667-b1c37355833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:28.431526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23af236-5f24-4660-9667-b1c37355833b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:28.432752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23af236-5f24-4660-9667-b1c37355833b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:28.434497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23af236-5f24-4660-9667-b1c37355833b admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:28.438959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23af236-5f24-4660-9667-b1c37355833b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:28.439143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23af236-5f24-4660-9667-b1c37355833b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:28.471944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 948/1910] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:44:28 2022] POST /identity/v3/users => generated 281 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:28.476179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd10dd-1554-44c4-9f3d-7839598731dd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:28.488556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd10dd-1554-44c4-9f3d-7839598731dd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:28.498901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4d409-5002-4939-8a49-13f38c9d0eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:28.499870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4d409-5002-4939-8a49-13f38c9d0eeb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:28.500542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4d409-5002-4939-8a49-13f38c9d0eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:28.500542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4d409-5002-4939-8a49-13f38c9d0eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:28.501628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4d409-5002-4939-8a49-13f38c9d0eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:28.504318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4d409-5002-4939-8a49-13f38c9d0eeb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:28.521165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 963/1911] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:28.526149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23af236-5f24-4660-9667-b1c37355833b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:28.538418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23af236-5f24-4660-9667-b1c37355833b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:28.544975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd483c4-c12e-4de6-9fa7-26e661ce9b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:28.545707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd483c4-c12e-4de6-9fa7-26e661ce9b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:28.546320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd483c4-c12e-4de6-9fa7-26e661ce9b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:28.546909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd483c4-c12e-4de6-9fa7-26e661ce9b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:28.548526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd483c4-c12e-4de6-9fa7-26e661ce9b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:28.553002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd483c4-c12e-4de6-9fa7-26e661ce9b43 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:28.568714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 949/1912] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:28.569913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4d409-5002-4939-8a49-13f38c9d0eeb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:28.617074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4d409-5002-4939-8a49-13f38c9d0eeb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:28.629179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85680ba-a390-41c0-8811-721a12f6bdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:28.634921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85680ba-a390-41c0-8811-721a12f6bdcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:28.636128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85680ba-a390-41c0-8811-721a12f6bdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:28.636623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85680ba-a390-41c0-8811-721a12f6bdcd admin admin] PATH_INFO: `/v3/projects/d023778e6ac646c7b1e1db266ee45847/users/e41939e7da48468f8c96dad83a19270f/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:28.669465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:28.674312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85680ba-a390-41c0-8811-721a12f6bdcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d023778e6ac646c7b1e1db266ee45847, user_id=e41939e7da48468f8c96dad83a19270f, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:28.694972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85680ba-a390-41c0-8811-721a12f6bdcd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:28.716141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 964/1913] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:28 2022] PUT /identity/v3/projects/d023778e6ac646c7b1e1db266ee45847/users/e41939e7da48468f8c96dad83a19270f/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:44:28.729372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0450fe-85a7-4f4a-8cba-e644467d35e8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:28.730107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0450fe-85a7-4f4a-8cba-e644467d35e8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:28.730584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0450fe-85a7-4f4a-8cba-e644467d35e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:28.791570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0d0450fe-85a7-4f4a-8cba-e644467d35e8 None None] MFA Rules not processed for user `e41939e7da48468f8c96dad83a19270f`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:28.797851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0d0450fe-85a7-4f4a-8cba-e644467d35e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:28.927796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 950/1914] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:28.979092 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85680ba-a390-41c0-8811-721a12f6bdcd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:28.982330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d85680ba-a390-41c0-8811-721a12f6bdcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:29.093272 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85680ba-a390-41c0-8811-721a12f6bdcd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:29.103325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929e690-6057-454a-aa28-999a38f6fc8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:29.104025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929e690-6057-454a-aa28-999a38f6fc8b service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:29.104315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929e690-6057-454a-aa28-999a38f6fc8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:29.104657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929e690-6057-454a-aa28-999a38f6fc8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:29.118707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929e690-6057-454a-aa28-999a38f6fc8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:29.123315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929e690-6057-454a-aa28-999a38f6fc8b service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:29.136204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 965/1915] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:29.157966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0450fe-85a7-4f4a-8cba-e644467d35e8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:29.277258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0450fe-85a7-4f4a-8cba-e644467d35e8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:29.284394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:29.285374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:29.285856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:29.286352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] PATH_INFO: `/v3/users/e41939e7da48468f8c96dad83a19270f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:29.287998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41939e7da48468f8c96dad83a19270f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:29.292616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:29.379801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41939e7da48468f8c96dad83a19270f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.380567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41939e7da48468f8c96dad83a19270f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.393033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41939e7da48468f8c96dad83a19270f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.400189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41939e7da48468f8c96dad83a19270f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.404290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41939e7da48468f8c96dad83a19270f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.410796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:29.438156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 951/1916] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:44:29 2022] DELETE /identity/v3/users/e41939e7da48468f8c96dad83a19270f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:29.440832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e6347-c618-4280-80c5-300c3869baf2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:29.441134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e6347-c618-4280-80c5-300c3869baf2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:29.441413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e6347-c618-4280-80c5-300c3869baf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:29.458485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:29.460085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:29.474293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-518e6347-c618-4280-80c5-300c3869baf2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:29.476068 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-518e6347-c618-4280-80c5-300c3869baf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:29.568238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5407e-0df9-4841-b566-2cbb48d8cb9d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:29.573578 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:29.574239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:29.574519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:29.574794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] PATH_INFO: `/v3/projects/8392e77cd14f48dfa8a4e65d5035a120` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:29.575947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8392e77cd14f48dfa8a4e65d5035a120)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:29.578047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:29.588528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 966/1917] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:29.641291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e6347-c618-4280-80c5-300c3869baf2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:29.645228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-518e6347-c618-4280-80c5-300c3869baf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:29.663127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8392e77cd14f48dfa8a4e65d5035a120', 'request_id': 'req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.674884 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8392e77cd14f48dfa8a4e65d5035a120', 'request_id': 'req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.684044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8392e77cd14f48dfa8a4e65d5035a120', 'request_id': 'req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.710698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] The token cache is being invalidate because project 8392e77cd14f48dfa8a4e65d5035a120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:44:29.712089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.714337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.720732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 952/1918] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:44:29 2022] DELETE /identity/v3/projects/8392e77cd14f48dfa8a4e65d5035a120 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:29.728849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:29.731002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:29.771973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e6347-c618-4280-80c5-300c3869baf2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:29.846287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e88a8-8f0e-4e5b-95f7-2abcb9e13adf admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:29.853683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f3a02-0333-41b5-8a28-0895c9ee31f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:29.854348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3a02-0333-41b5-8a28-0895c9ee31f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:29.854707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3a02-0333-41b5-8a28-0895c9ee31f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:29.861928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3a02-0333-41b5-8a28-0895c9ee31f6 admin admin] PATH_INFO: `/v3/users/47b06ddebb76462aa75f802e833ba179` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:29.861928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f3a02-0333-41b5-8a28-0895c9ee31f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b06ddebb76462aa75f802e833ba179)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:29.861928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f3a02-0333-41b5-8a28-0895c9ee31f6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:29.885083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:29.903239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16929445-dfc2-4932-94bc-1476f4bf7239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:29.903239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16929445-dfc2-4932-94bc-1476f4bf7239 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:29.903239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16929445-dfc2-4932-94bc-1476f4bf7239 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:29.905863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16929445-dfc2-4932-94bc-1476f4bf7239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:29.912019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-16929445-dfc2-4932-94bc-1476f4bf7239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:29.932580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:29.941334 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-223f3a02-0333-41b5-8a28-0895c9ee31f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b06ddebb76462aa75f802e833ba179'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.942311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-223f3a02-0333-41b5-8a28-0895c9ee31f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b06ddebb76462aa75f802e833ba179'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.946729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16929445-dfc2-4932-94bc-1476f4bf7239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:29.949752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16929445-dfc2-4932-94bc-1476f4bf7239 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:29.963522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-223f3a02-0333-41b5-8a28-0895c9ee31f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b06ddebb76462aa75f802e833ba179'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.975924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 967/1919] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:29.983688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-223f3a02-0333-41b5-8a28-0895c9ee31f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b06ddebb76462aa75f802e833ba179'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:29.984803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-223f3a02-0333-41b5-8a28-0895c9ee31f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b06ddebb76462aa75f802e833ba179'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:30.007288 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 953/1920] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:44:29 2022] DELETE /identity/v3/users/47b06ddebb76462aa75f802e833ba179 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:30.481918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16929445-dfc2-4932-94bc-1476f4bf7239 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:30.481918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-16929445-dfc2-4932-94bc-1476f4bf7239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:30.502138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16929445-dfc2-4932-94bc-1476f4bf7239 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:30.593801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:30.594628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:30.594938 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:30.595318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] PATH_INFO: `/v3/projects/d023778e6ac646c7b1e1db266ee45847` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:30.597088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d023778e6ac646c7b1e1db266ee45847)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:30.599136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:30.676868 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd023778e6ac646c7b1e1db266ee45847', 'request_id': 'req-82210874-f6fa-4b72-acb3-4a83ab3f963d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:30.693347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd023778e6ac646c7b1e1db266ee45847', 'request_id': 'req-82210874-f6fa-4b72-acb3-4a83ab3f963d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:30.702168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd023778e6ac646c7b1e1db266ee45847', 'request_id': 'req-82210874-f6fa-4b72-acb3-4a83ab3f963d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:30.724866 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] The token cache is being invalidate because project d023778e6ac646c7b1e1db266ee45847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:44:30.725219 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:30.725887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:30.727397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 968/1921] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:44:30 2022] DELETE /identity/v3/projects/d023778e6ac646c7b1e1db266ee45847 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:30.732369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91825403-3cce-463d-bd7c-f65a68263833 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:30.732703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91825403-3cce-463d-bd7c-f65a68263833 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:30.733078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91825403-3cce-463d-bd7c-f65a68263833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:30.756805 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-91825403-3cce-463d-bd7c-f65a68263833 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:30.758161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-91825403-3cce-463d-bd7c-f65a68263833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:30.801086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:30.801086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:30.860267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 954/1922] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:30.869336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a70ac-d022-4814-9808-244a7336c9d5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:30.869813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a70ac-d022-4814-9808-244a7336c9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:30.870096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a70ac-d022-4814-9808-244a7336c9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:30.895910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82210874-f6fa-4b72-acb3-4a83ab3f963d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:30.900648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:30.901280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:30.901555 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:30.901823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] PATH_INFO: `/v3/projects/ea02af9022264c3e83f75edddc9951fa` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:30.902582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6b6a70ac-d022-4814-9808-244a7336c9d5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:30.902922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea02af9022264c3e83f75edddc9951fa)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:30.903822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6b6a70ac-d022-4814-9808-244a7336c9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:30.904867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:30.913247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 955/1923] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:44:30.917384 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a70ac-d022-4814-9808-244a7336c9d5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:30.926647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a70ac-d022-4814-9808-244a7336c9d5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:30.931358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809f890-2b02-47ba-a9ce-21971431d80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:30.931961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809f890-2b02-47ba-a9ce-21971431d80f admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:30.932246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809f890-2b02-47ba-a9ce-21971431d80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:30.932551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809f890-2b02-47ba-a9ce-21971431d80f admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:30.933201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809f890-2b02-47ba-a9ce-21971431d80f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:44:30.933509 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809f890-2b02-47ba-a9ce-21971431d80f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:30.935116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809f890-2b02-47ba-a9ce-21971431d80f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:30.946097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:30.948474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 956/1924] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:44:30.952804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809f890-2b02-47ba-a9ce-21971431d80f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:30.964202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809f890-2b02-47ba-a9ce-21971431d80f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:30.965714 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea02af9022264c3e83f75edddc9951fa', 'request_id': 'req-de053017-f283-4348-ad3b-5a79d6d03660'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:30.973332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea02af9022264c3e83f75edddc9951fa', 'request_id': 'req-de053017-f283-4348-ad3b-5a79d6d03660'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:30.979198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea02af9022264c3e83f75edddc9951fa', 'request_id': 'req-de053017-f283-4348-ad3b-5a79d6d03660'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:30.992774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] The token cache is being invalidate because project ea02af9022264c3e83f75edddc9951fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:44:30.993064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:30.993765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-de053017-f283-4348-ad3b-5a79d6d03660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:30.997028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 969/1925] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:44:30 2022] DELETE /identity/v3/projects/ea02af9022264c3e83f75edddc9951fa => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:31.000631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998aba5-2446-49c0-8281-247aca3075b6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.000947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998aba5-2446-49c0-8281-247aca3075b6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.001225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998aba5-2446-49c0-8281-247aca3075b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.021126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7998aba5-2446-49c0-8281-247aca3075b6 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:31.022348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7998aba5-2446-49c0-8281-247aca3075b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:31.035555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fc4e9-e8e2-41e5-a341-b95b86835fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.036193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fc4e9-e8e2-41e5-a341-b95b86835fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.036464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fc4e9-e8e2-41e5-a341-b95b86835fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.036788 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fc4e9-e8e2-41e5-a341-b95b86835fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.037429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fc4e9-e8e2-41e5-a341-b95b86835fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.039354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fc4e9-e8e2-41e5-a341-b95b86835fa3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.063621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:31.078382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 957/1926] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:44:30 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:31.079651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fc4e9-e8e2-41e5-a341-b95b86835fa3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:31.085834 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-405fc4e9-e8e2-41e5-a341-b95b86835fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:31.104710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 970/1927] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:44:31.113203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32ce84-f196-4209-9d25-6141b1a04ad1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.113490 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32ce84-f196-4209-9d25-6141b1a04ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.113768 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32ce84-f196-4209-9d25-6141b1a04ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.134884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bf32ce84-f196-4209-9d25-6141b1a04ad1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:31.136085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bf32ce84-f196-4209-9d25-6141b1a04ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:31.161822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fc4e9-e8e2-41e5-a341-b95b86835fa3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:31.169101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d532cf6-f270-482c-ad39-4dcef774eb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.169999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d532cf6-f270-482c-ad39-4dcef774eb24 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.170466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d532cf6-f270-482c-ad39-4dcef774eb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.170936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d532cf6-f270-482c-ad39-4dcef774eb24 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.172021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d532cf6-f270-482c-ad39-4dcef774eb24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.175287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d532cf6-f270-482c-ad39-4dcef774eb24 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.206362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 958/1928] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:44:31 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:31.211980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d532cf6-f270-482c-ad39-4dcef774eb24 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:31.225668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d532cf6-f270-482c-ad39-4dcef774eb24 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:31.228644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 971/1929] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:31.237518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ca1fb-42c6-4037-871d-7f6bbed6911a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.238157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ca1fb-42c6-4037-871d-7f6bbed6911a admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.238439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ca1fb-42c6-4037-871d-7f6bbed6911a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.238718 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ca1fb-42c6-4037-871d-7f6bbed6911a admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.239378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ca1fb-42c6-4037-871d-7f6bbed6911a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.241083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ca1fb-42c6-4037-871d-7f6bbed6911a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.243938 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32ce84-f196-4209-9d25-6141b1a04ad1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:31.256739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32ce84-f196-4209-9d25-6141b1a04ad1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:31.266005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e23f8c-fbbd-4e0c-840d-54991f28e402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.266005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e23f8c-fbbd-4e0c-840d-54991f28e402 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.266005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e23f8c-fbbd-4e0c-840d-54991f28e402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.266005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e23f8c-fbbd-4e0c-840d-54991f28e402 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.270005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e23f8c-fbbd-4e0c-840d-54991f28e402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:44:31.270005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e23f8c-fbbd-4e0c-840d-54991f28e402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.270526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 959/1930] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:31.271296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e23f8c-fbbd-4e0c-840d-54991f28e402 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.271855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ca1fb-42c6-4037-871d-7f6bbed6911a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:31.284933 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ca1fb-42c6-4037-871d-7f6bbed6911a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:31.290084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 972/1931] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:44:31.291517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006289c-089f-4e51-8f1b-a9faa8117e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.292350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006289c-089f-4e51-8f1b-a9faa8117e9d admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.292851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006289c-089f-4e51-8f1b-a9faa8117e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.293405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006289c-089f-4e51-8f1b-a9faa8117e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.294469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006289c-089f-4e51-8f1b-a9faa8117e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.294771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e23f8c-fbbd-4e0c-840d-54991f28e402 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:31.296609 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006289c-089f-4e51-8f1b-a9faa8117e9d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.305968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e23f8c-fbbd-4e0c-840d-54991f28e402 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:31.314135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 960/1932] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:31.315994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006289c-089f-4e51-8f1b-a9faa8117e9d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:31.316994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d26936-556b-4c95-a007-156633aa3ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.317621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d26936-556b-4c95-a007-156633aa3ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.318309 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d26936-556b-4c95-a007-156633aa3ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.318586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d26936-556b-4c95-a007-156633aa3ac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.319232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d26936-556b-4c95-a007-156633aa3ac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.321517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d26936-556b-4c95-a007-156633aa3ac7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.334821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006289c-089f-4e51-8f1b-a9faa8117e9d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:31.343513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d93d0-22d6-4253-b7fd-408f96d552a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.344475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d93d0-22d6-4253-b7fd-408f96d552a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.344960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d93d0-22d6-4253-b7fd-408f96d552a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.345701 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d93d0-22d6-4253-b7fd-408f96d552a5 admin admin] PATH_INFO: `/v3/projects/6f19f3225f6e43fb8534aa2b1087273d/users/3f02160b6f634af9b4f4c7f3e7810971/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.376439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 973/1933] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:44:31 2022] POST /identity/v3/projects => generated 381 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:31.377818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d26936-556b-4c95-a007-156633aa3ac7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:31.380357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d93d0-22d6-4253-b7fd-408f96d552a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f19f3225f6e43fb8534aa2b1087273d, user_id=3f02160b6f634af9b4f4c7f3e7810971, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.383983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d93d0-22d6-4253-b7fd-408f96d552a5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.416745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 961/1934] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:31 2022] PUT /identity/v3/projects/6f19f3225f6e43fb8534aa2b1087273d/users/3f02160b6f634af9b4f4c7f3e7810971/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:31.427989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd88a67-b544-4920-8ec5-bbeb15e22d2b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.428278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd88a67-b544-4920-8ec5-bbeb15e22d2b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.428698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd88a67-b544-4920-8ec5-bbeb15e22d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.436113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:31.477539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ebd88a67-b544-4920-8ec5-bbeb15e22d2b None None] MFA Rules not processed for user `3f02160b6f634af9b4f4c7f3e7810971`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:31.479179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ebd88a67-b544-4920-8ec5-bbeb15e22d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:31.500133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d26936-556b-4c95-a007-156633aa3ac7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:31.584070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288e615-f7f6-48bb-9949-4fd74a697a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.584930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e615-f7f6-48bb-9949-4fd74a697a15 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.585317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e615-f7f6-48bb-9949-4fd74a697a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.585697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e615-f7f6-48bb-9949-4fd74a697a15 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.586637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288e615-f7f6-48bb-9949-4fd74a697a15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.589875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288e615-f7f6-48bb-9949-4fd74a697a15 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.590988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 962/1935] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:31.606121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd88a67-b544-4920-8ec5-bbeb15e22d2b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:31.609827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ebd88a67-b544-4920-8ec5-bbeb15e22d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:31.633533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 974/1936] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:44:31 2022] POST /identity/v3/users => generated 291 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:31.643701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288e615-f7f6-48bb-9949-4fd74a697a15 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:31.655936 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288e615-f7f6-48bb-9949-4fd74a697a15 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:31.661719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150cf87-675b-4e51-b61b-7861059c0427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.662442 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150cf87-675b-4e51-b61b-7861059c0427 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.662827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150cf87-675b-4e51-b61b-7861059c0427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.663166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150cf87-675b-4e51-b61b-7861059c0427 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.663925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150cf87-675b-4e51-b61b-7861059c0427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.665813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150cf87-675b-4e51-b61b-7861059c0427 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.672112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:31.685499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 975/1937] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:44:31.691291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150cf87-675b-4e51-b61b-7861059c0427 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:31.710139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150cf87-675b-4e51-b61b-7861059c0427 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:31.710999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd88a67-b544-4920-8ec5-bbeb15e22d2b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:31.715167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac3f66-f4dd-4000-8441-bde4415da0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.715772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac3f66-f4dd-4000-8441-bde4415da0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.716051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac3f66-f4dd-4000-8441-bde4415da0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.716322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac3f66-f4dd-4000-8441-bde4415da0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.716994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac3f66-f4dd-4000-8441-bde4415da0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.718578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac3f66-f4dd-4000-8441-bde4415da0c2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.719483 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433762dc-ac14-46ae-9909-da3d440425ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.720381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433762dc-ac14-46ae-9909-da3d440425ae service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.721269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433762dc-ac14-46ae-9909-da3d440425ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.721773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433762dc-ac14-46ae-9909-da3d440425ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.731336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 976/1938] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:31.735269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433762dc-ac14-46ae-9909-da3d440425ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.737598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433762dc-ac14-46ae-9909-da3d440425ae service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.738852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac3f66-f4dd-4000-8441-bde4415da0c2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:31.752644 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 963/1939] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:31.755604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac3f66-f4dd-4000-8441-bde4415da0c2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:31.760830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b944f-8038-42f0-8de9-1cbf23850bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:31.761717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b944f-8038-42f0-8de9-1cbf23850bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.762146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b944f-8038-42f0-8de9-1cbf23850bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.762566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b944f-8038-42f0-8de9-1cbf23850bbb admin admin] PATH_INFO: `/v3/projects/f22dc9d7e35c4e3daf39ff12133876da/users/adff6db3b2c848378fc32a3b7c65d27f/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.784109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b944f-8038-42f0-8de9-1cbf23850bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22dc9d7e35c4e3daf39ff12133876da, user_id=adff6db3b2c848378fc32a3b7c65d27f, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:31.786834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b944f-8038-42f0-8de9-1cbf23850bbb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:31.808933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 977/1940] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:31 2022] PUT /identity/v3/projects/f22dc9d7e35c4e3daf39ff12133876da/users/adff6db3b2c848378fc32a3b7c65d27f/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:31.821258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25282de-e5e4-4c35-86b2-76bb9d940a05 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:31.821544 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25282de-e5e4-4c35-86b2-76bb9d940a05 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:31.821826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25282de-e5e4-4c35-86b2-76bb9d940a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:31.859515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a25282de-e5e4-4c35-86b2-76bb9d940a05 None None] MFA Rules not processed for user `adff6db3b2c848378fc32a3b7c65d27f`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:31.861651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a25282de-e5e4-4c35-86b2-76bb9d940a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:31.947723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 964/1941] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:31.952325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b944f-8038-42f0-8de9-1cbf23850bbb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:32.027444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b944f-8038-42f0-8de9-1cbf23850bbb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:32.032321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b6573-f70c-43ab-bb81-1cd70c2735b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:32.032995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b6573-f70c-43ab-bb81-1cd70c2735b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:32.033282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b6573-f70c-43ab-bb81-1cd70c2735b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:32.033557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b6573-f70c-43ab-bb81-1cd70c2735b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:32.034200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b6573-f70c-43ab-bb81-1cd70c2735b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:32.036185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b6573-f70c-43ab-bb81-1cd70c2735b6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:32.055680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:32.074813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 978/1942] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:44:31 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:44:32.076254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25282de-e5e4-4c35-86b2-76bb9d940a05 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:32.091536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25282de-e5e4-4c35-86b2-76bb9d940a05 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:32.183489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecee384-d5e3-48cd-934f-245c7ae58210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:32.184773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecee384-d5e3-48cd-934f-245c7ae58210 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:32.185401 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecee384-d5e3-48cd-934f-245c7ae58210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:32.185930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecee384-d5e3-48cd-934f-245c7ae58210 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:32.187653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecee384-d5e3-48cd-934f-245c7ae58210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:32.188833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecee384-d5e3-48cd-934f-245c7ae58210 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:32.217949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 965/1943] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:44:32 2022] POST /identity/v3/users => generated 297 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:32.222143 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b6573-f70c-43ab-bb81-1cd70c2735b6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:32.234386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b6573-f70c-43ab-bb81-1cd70c2735b6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:32.241917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1c3a3-95f0-476e-97d4-bdd34cabab6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:32.242532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1c3a3-95f0-476e-97d4-bdd34cabab6b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:32.242808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1c3a3-95f0-476e-97d4-bdd34cabab6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:32.243078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1c3a3-95f0-476e-97d4-bdd34cabab6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:32.243803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1c3a3-95f0-476e-97d4-bdd34cabab6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:32.245430 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1c3a3-95f0-476e-97d4-bdd34cabab6b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:32.261357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 979/1944] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:44:32.262646 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecee384-d5e3-48cd-934f-245c7ae58210 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:32.274195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecee384-d5e3-48cd-934f-245c7ae58210 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:32.279489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e673da0-1db5-47ba-947f-bcb4c45a1989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:32.280277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e673da0-1db5-47ba-947f-bcb4c45a1989 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:32.280697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e673da0-1db5-47ba-947f-bcb4c45a1989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:32.281160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e673da0-1db5-47ba-947f-bcb4c45a1989 admin admin] PATH_INFO: `/v3/projects/46922c0fcee54413bf9627426969b807/users/32f7335665dc4877b16945d49d966995/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:32.297011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e673da0-1db5-47ba-947f-bcb4c45a1989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46922c0fcee54413bf9627426969b807, user_id=32f7335665dc4877b16945d49d966995, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:32.301022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e673da0-1db5-47ba-947f-bcb4c45a1989 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:32.317541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 966/1945] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:32 2022] PUT /identity/v3/projects/46922c0fcee54413bf9627426969b807/users/32f7335665dc4877b16945d49d966995/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:32.321543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1c3a3-95f0-476e-97d4-bdd34cabab6b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:32.332723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1c3a3-95f0-476e-97d4-bdd34cabab6b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:32.397259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5437ca-1688-43cd-bab1-6be2ba61a161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:32.397868 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5437ca-1688-43cd-bab1-6be2ba61a161 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:32.398147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5437ca-1688-43cd-bab1-6be2ba61a161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:32.398469 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5437ca-1688-43cd-bab1-6be2ba61a161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:32.399122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5437ca-1688-43cd-bab1-6be2ba61a161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:32.400960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5437ca-1688-43cd-bab1-6be2ba61a161 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:32.411235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 980/1946] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:44:32.416897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e673da0-1db5-47ba-947f-bcb4c45a1989 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:32.427857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e673da0-1db5-47ba-947f-bcb4c45a1989 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:32.432368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4921158-56d9-46bf-a315-4dcc57558110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:32.433054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4921158-56d9-46bf-a315-4dcc57558110 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:32.433332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4921158-56d9-46bf-a315-4dcc57558110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:32.433607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4921158-56d9-46bf-a315-4dcc57558110 admin admin] PATH_INFO: `/v3/projects/46922c0fcee54413bf9627426969b807/users/32f7335665dc4877b16945d49d966995/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:32.435526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4921158-56d9-46bf-a315-4dcc57558110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46922c0fcee54413bf9627426969b807, user_id=32f7335665dc4877b16945d49d966995, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:32.438186 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4921158-56d9-46bf-a315-4dcc57558110 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:32.449092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 967/1947] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:32 2022] PUT /identity/v3/projects/46922c0fcee54413bf9627426969b807/users/32f7335665dc4877b16945d49d966995/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:32.459343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25733b9-0ae5-49aa-8f61-5cbb66516918 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:32.459600 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25733b9-0ae5-49aa-8f61-5cbb66516918 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:32.459882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25733b9-0ae5-49aa-8f61-5cbb66516918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:32.490430 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a25733b9-0ae5-49aa-8f61-5cbb66516918 None None] MFA Rules not processed for user `32f7335665dc4877b16945d49d966995`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:32.491670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a25733b9-0ae5-49aa-8f61-5cbb66516918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:32.563546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 981/1948] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:32.575677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4921158-56d9-46bf-a315-4dcc57558110 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:32.599117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:32.635530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4921158-56d9-46bf-a315-4dcc57558110 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:32.639247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dad6e4-ee89-4da6-a248-3befe0e9e7cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:32.639869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dad6e4-ee89-4da6-a248-3befe0e9e7cf service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:32.640163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dad6e4-ee89-4da6-a248-3befe0e9e7cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:32.640428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dad6e4-ee89-4da6-a248-3befe0e9e7cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:32.650643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dad6e4-ee89-4da6-a248-3befe0e9e7cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:32.652092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dad6e4-ee89-4da6-a248-3befe0e9e7cf service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:32.678720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 968/1949] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:32.759532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25733b9-0ae5-49aa-8f61-5cbb66516918 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:32.770499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25733b9-0ae5-49aa-8f61-5cbb66516918 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:32.774807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9746f-93c2-4d78-b6fa-c63a01ca8173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:32.775143 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9746f-93c2-4d78-b6fa-c63a01ca8173 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:32.775419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9746f-93c2-4d78-b6fa-c63a01ca8173 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:32.775690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9746f-93c2-4d78-b6fa-c63a01ca8173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:32.848149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9746f-93c2-4d78-b6fa-c63a01ca8173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:32.848149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9746f-93c2-4d78-b6fa-c63a01ca8173 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:32.855152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 982/1950] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:34.982532 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dad6e4-ee89-4da6-a248-3befe0e9e7cf service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:35.059093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dad6e4-ee89-4da6-a248-3befe0e9e7cf service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:35.067339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:35.068117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:35.068480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:35.068849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] PATH_INFO: `/v3/users/3f02160b6f634af9b4f4c7f3e7810971` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:35.070123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f02160b6f634af9b4f4c7f3e7810971)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:35.072637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:35.166871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f02160b6f634af9b4f4c7f3e7810971'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:35.168274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f02160b6f634af9b4f4c7f3e7810971'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:35.177696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f02160b6f634af9b4f4c7f3e7810971'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:35.186423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f02160b6f634af9b4f4c7f3e7810971'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:35.187102 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f02160b6f634af9b4f4c7f3e7810971'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:35.210675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 969/1951] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:44:34 2022] DELETE /identity/v3/users/3f02160b6f634af9b4f4c7f3e7810971 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:35.215994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58783-c4b6-488f-92d0-9af3552b0c77 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:35.216475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58783-c4b6-488f-92d0-9af3552b0c77 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:35.216583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58783-c4b6-488f-92d0-9af3552b0c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:35.226287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:35.240541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2cc58783-c4b6-488f-92d0-9af3552b0c77 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:35.241776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2cc58783-c4b6-488f-92d0-9af3552b0c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:35.354528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 983/1952] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:35.393473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:35.395069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:35.463967 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:35.478954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b90d7-adae-4397-ad57-34fe85f60019 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:35.484998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc80688-ae31-4300-9420-6b706c8b5c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:35.485578 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80688-ae31-4300-9420-6b706c8b5c62 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:35.487269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80688-ae31-4300-9420-6b706c8b5c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:35.487624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80688-ae31-4300-9420-6b706c8b5c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:35.503719 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc80688-ae31-4300-9420-6b706c8b5c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:35.505251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc80688-ae31-4300-9420-6b706c8b5c62 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:35.520026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 970/1953] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:36.313932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc58783-c4b6-488f-92d0-9af3552b0c77 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:36.317973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2cc58783-c4b6-488f-92d0-9af3552b0c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:36.332168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc58783-c4b6-488f-92d0-9af3552b0c77 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:36.337059 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:36.337710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:36.337992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:36.338274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] PATH_INFO: `/v3/projects/6f19f3225f6e43fb8534aa2b1087273d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:36.339411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f19f3225f6e43fb8534aa2b1087273d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:36.341362 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:36.407132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f19f3225f6e43fb8534aa2b1087273d', 'request_id': 'req-0d243ff9-1ad7-46af-af29-28e834828af4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:36.415136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f19f3225f6e43fb8534aa2b1087273d', 'request_id': 'req-0d243ff9-1ad7-46af-af29-28e834828af4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:36.423873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f19f3225f6e43fb8534aa2b1087273d', 'request_id': 'req-0d243ff9-1ad7-46af-af29-28e834828af4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:36.446918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] The token cache is being invalidate because project 6f19f3225f6e43fb8534aa2b1087273d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:44:36.447307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:36.448255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0d243ff9-1ad7-46af-af29-28e834828af4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:36.466198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 984/1954] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:44:36 2022] DELETE /identity/v3/projects/6f19f3225f6e43fb8534aa2b1087273d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:45.514158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=8 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:46.439521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:47.284744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc80688-ae31-4300-9420-6b706c8b5c62 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:47.286564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0dc80688-ae31-4300-9420-6b706c8b5c62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:47.299385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc80688-ae31-4300-9420-6b706c8b5c62 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:47.369943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:47.370582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:47.371573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:47.371857 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] PATH_INFO: `/v3/users/adff6db3b2c848378fc32a3b7c65d27f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:47.373323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adff6db3b2c848378fc32a3b7c65d27f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:47.375477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:47.445695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adff6db3b2c848378fc32a3b7c65d27f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:47.446559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adff6db3b2c848378fc32a3b7c65d27f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:47.456764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adff6db3b2c848378fc32a3b7c65d27f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:47.464142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adff6db3b2c848378fc32a3b7c65d27f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:47.465028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adff6db3b2c848378fc32a3b7c65d27f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:47.489313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 971/1955] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:44:47 2022] DELETE /identity/v3/users/adff6db3b2c848378fc32a3b7c65d27f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:44:47.491156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894281b8-d630-4fa3-98d2-f12409b72b8d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:47.491649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894281b8-d630-4fa3-98d2-f12409b72b8d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:47.491739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894281b8-d630-4fa3-98d2-f12409b72b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:47.514173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-894281b8-d630-4fa3-98d2-f12409b72b8d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:47.515457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-894281b8-d630-4fa3-98d2-f12409b72b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:47.582337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 985/1956] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:47.592050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:47.593516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:47.653816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca0749-8227-4cad-a071-ccb5a8bac330 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:47.657675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e64b1c-2455-463d-8319-e69cf8e5e0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:47.658287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e64b1c-2455-463d-8319-e69cf8e5e0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:47.658565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e64b1c-2455-463d-8319-e69cf8e5e0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:47.658835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e64b1c-2455-463d-8319-e69cf8e5e0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:47.668160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e64b1c-2455-463d-8319-e69cf8e5e0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:47.669631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e64b1c-2455-463d-8319-e69cf8e5e0e5 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:47.681291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 972/1957] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:47.976542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894281b8-d630-4fa3-98d2-f12409b72b8d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:47.978017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-894281b8-d630-4fa3-98d2-f12409b72b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:47.990860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894281b8-d630-4fa3-98d2-f12409b72b8d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:47.995396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:47.996033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:47.996314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:47.996631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] PATH_INFO: `/v3/projects/f22dc9d7e35c4e3daf39ff12133876da` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:47.997744 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f22dc9d7e35c4e3daf39ff12133876da)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:47.999655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:48.049302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f22dc9d7e35c4e3daf39ff12133876da', 'request_id': 'req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.056282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f22dc9d7e35c4e3daf39ff12133876da', 'request_id': 'req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.062010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f22dc9d7e35c4e3daf39ff12133876da', 'request_id': 'req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.075854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] The token cache is being invalidate because project f22dc9d7e35c4e3daf39ff12133876da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:44:48.076138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.076788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.078280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 986/1958] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:44:47 2022] DELETE /identity/v3/projects/f22dc9d7e35c4e3daf39ff12133876da => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:48.082194 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e64b1c-2455-463d-8319-e69cf8e5e0e5 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:48.092081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e64b1c-2455-463d-8319-e69cf8e5e0e5 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:48.111764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:48.159084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ee1f-dfdb-48fa-a512-4fdf8363d37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:48.159758 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ee1f-dfdb-48fa-a512-4fdf8363d37a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:48.160715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ee1f-dfdb-48fa-a512-4fdf8363d37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:48.161460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ee1f-dfdb-48fa-a512-4fdf8363d37a admin admin] PATH_INFO: `/v3/users/32f7335665dc4877b16945d49d966995` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:48.162721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ee1f-dfdb-48fa-a512-4fdf8363d37a admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f7335665dc4877b16945d49d966995)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:48.164716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ee1f-dfdb-48fa-a512-4fdf8363d37a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:48.212910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-50e9ee1f-dfdb-48fa-a512-4fdf8363d37a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f7335665dc4877b16945d49d966995'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.213571 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-50e9ee1f-dfdb-48fa-a512-4fdf8363d37a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f7335665dc4877b16945d49d966995'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.220301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-50e9ee1f-dfdb-48fa-a512-4fdf8363d37a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f7335665dc4877b16945d49d966995'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.228722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-50e9ee1f-dfdb-48fa-a512-4fdf8363d37a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f7335665dc4877b16945d49d966995'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.229467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-50e9ee1f-dfdb-48fa-a512-4fdf8363d37a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f7335665dc4877b16945d49d966995'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.247418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 973/1959] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:44:48 2022] DELETE /identity/v3/users/32f7335665dc4877b16945d49d966995 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:48.540333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:48.541925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:48.557203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c03e8e-394e-4c31-9cae-a74970e8ce2e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:48.623253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:48.624074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:48.624352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:48.624761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] PATH_INFO: `/v3/projects/46922c0fcee54413bf9627426969b807` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:48.626081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] RBAC: Authorizing `identity:delete_project(project_id=46922c0fcee54413bf9627426969b807)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:48.630001 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:48.662818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:48.685623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46922c0fcee54413bf9627426969b807', 'request_id': 'req-b4ca11ec-9d63-4f3f-9beb-b124148f380c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.694122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46922c0fcee54413bf9627426969b807', 'request_id': 'req-b4ca11ec-9d63-4f3f-9beb-b124148f380c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.700252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46922c0fcee54413bf9627426969b807', 'request_id': 'req-b4ca11ec-9d63-4f3f-9beb-b124148f380c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.713120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] The token cache is being invalidate because project 46922c0fcee54413bf9627426969b807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:44:48.713566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.714291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4ca11ec-9d63-4f3f-9beb-b124148f380c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:48.716573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 987/1960] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:44:48 2022] DELETE /identity/v3/projects/46922c0fcee54413bf9627426969b807 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:48.721609 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8f2d4-ee2b-4e30-b514-8155a42f0143 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:48.722351 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8f2d4-ee2b-4e30-b514-8155a42f0143 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:48.722679 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8f2d4-ee2b-4e30-b514-8155a42f0143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:48.744792 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ffa8f2d4-ee2b-4e30-b514-8155a42f0143 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:48.746198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ffa8f2d4-ee2b-4e30-b514-8155a42f0143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:48.834919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 974/1961] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:44:48.842985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749271d4-5677-4d1e-bdf0-f3fbf5878f68 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:48.843280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749271d4-5677-4d1e-bdf0-f3fbf5878f68 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:48.843558 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749271d4-5677-4d1e-bdf0-f3fbf5878f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:48.865346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-749271d4-5677-4d1e-bdf0-f3fbf5878f68 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:48.866556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-749271d4-5677-4d1e-bdf0-f3fbf5878f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:48.874569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 988/1962] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:48.879116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8f2d4-ee2b-4e30-b514-8155a42f0143 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:48.888909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8f2d4-ee2b-4e30-b514-8155a42f0143 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:48.893630 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becc502-6be2-4cea-89e3-26fb90980b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:48.894359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becc502-6be2-4cea-89e3-26fb90980b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:48.894659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becc502-6be2-4cea-89e3-26fb90980b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:48.895023 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becc502-6be2-4cea-89e3-26fb90980b8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:48.895802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becc502-6be2-4cea-89e3-26fb90980b8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:44:48.896193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becc502-6be2-4cea-89e3-26fb90980b8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:48.897856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becc502-6be2-4cea-89e3-26fb90980b8f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:48.909358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 975/1963] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:44:48.915515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749271d4-5677-4d1e-bdf0-f3fbf5878f68 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:48.924889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749271d4-5677-4d1e-bdf0-f3fbf5878f68 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:48.930174 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd983831-e80f-48c4-90d3-c3d0d8b61e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:48.930835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd983831-e80f-48c4-90d3-c3d0d8b61e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:48.931118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd983831-e80f-48c4-90d3-c3d0d8b61e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:48.931390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd983831-e80f-48c4-90d3-c3d0d8b61e34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:48.932016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd983831-e80f-48c4-90d3-c3d0d8b61e34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:48.933955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd983831-e80f-48c4-90d3-c3d0d8b61e34 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:48.960585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 989/1964] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:44:48 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:48.964939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becc502-6be2-4cea-89e3-26fb90980b8f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:48.976230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becc502-6be2-4cea-89e3-26fb90980b8f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:49.003677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:49.036925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fc166-b8c7-4f4b-92a0-d95dafa21143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:49.037534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fc166-b8c7-4f4b-92a0-d95dafa21143 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.037811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fc166-b8c7-4f4b-92a0-d95dafa21143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.038091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fc166-b8c7-4f4b-92a0-d95dafa21143 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.038746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fc166-b8c7-4f4b-92a0-d95dafa21143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:49.040706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fc166-b8c7-4f4b-92a0-d95dafa21143 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:49.062274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 976/1965] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:44:48 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:44:49.066766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd983831-e80f-48c4-90d3-c3d0d8b61e34 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:49.077194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd983831-e80f-48c4-90d3-c3d0d8b61e34 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:49.081751 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea5058-0c52-4a06-9066-141a5eff097c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:49.082374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea5058-0c52-4a06-9066-141a5eff097c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.082653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea5058-0c52-4a06-9066-141a5eff097c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.082927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea5058-0c52-4a06-9066-141a5eff097c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.083565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea5058-0c52-4a06-9066-141a5eff097c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:49.085242 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea5058-0c52-4a06-9066-141a5eff097c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:49.097171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 990/1966] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:49.102070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fc166-b8c7-4f4b-92a0-d95dafa21143 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:49.112742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fc166-b8c7-4f4b-92a0-d95dafa21143 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:49.118065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a2ecf-b02b-4963-b264-1f28f7fd8c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:49.118694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2ecf-b02b-4963-b264-1f28f7fd8c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.118993 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2ecf-b02b-4963-b264-1f28f7fd8c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.119311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2ecf-b02b-4963-b264-1f28f7fd8c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.119992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a2ecf-b02b-4963-b264-1f28f7fd8c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:49.121716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a2ecf-b02b-4963-b264-1f28f7fd8c1b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:49.133535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 977/1967] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:44:49.137771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea5058-0c52-4a06-9066-141a5eff097c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:49.148996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea5058-0c52-4a06-9066-141a5eff097c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:49.153821 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c8452-8ea5-487a-8f78-f5c4762bf38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:49.154502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c8452-8ea5-487a-8f78-f5c4762bf38c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.154801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c8452-8ea5-487a-8f78-f5c4762bf38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.155118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c8452-8ea5-487a-8f78-f5c4762bf38c admin admin] PATH_INFO: `/v3/projects/ee994e59862847aabde2531555b48251/users/52ba88535aba4163b12f87aa0fcb838e/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.170719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c8452-8ea5-487a-8f78-f5c4762bf38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee994e59862847aabde2531555b48251, user_id=52ba88535aba4163b12f87aa0fcb838e, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:49.173417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c8452-8ea5-487a-8f78-f5c4762bf38c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:49.186729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 991/1968] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:49 2022] PUT /identity/v3/projects/ee994e59862847aabde2531555b48251/users/52ba88535aba4163b12f87aa0fcb838e/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:44:49.197081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d6080-4034-434d-b116-50da064168a7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.197373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d6080-4034-434d-b116-50da064168a7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.197653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d6080-4034-434d-b116-50da064168a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.228470 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f46d6080-4034-434d-b116-50da064168a7 None None] MFA Rules not processed for user `52ba88535aba4163b12f87aa0fcb838e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:49.229912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f46d6080-4034-434d-b116-50da064168a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:49.303686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 978/1969] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:49.308122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c8452-8ea5-487a-8f78-f5c4762bf38c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:49.373426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c8452-8ea5-487a-8f78-f5c4762bf38c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:49.378139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236e390-cf6c-4f94-baa8-8804f016f52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:49.378772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236e390-cf6c-4f94-baa8-8804f016f52a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.379072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236e390-cf6c-4f94-baa8-8804f016f52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.379329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236e390-cf6c-4f94-baa8-8804f016f52a admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.379967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236e390-cf6c-4f94-baa8-8804f016f52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:49.381895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236e390-cf6c-4f94-baa8-8804f016f52a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:49.408274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 992/1970] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:44:49 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:49.413070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d6080-4034-434d-b116-50da064168a7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:49.423969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d6080-4034-434d-b116-50da064168a7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:49.488823 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b90b2-8f1f-4ad5-baaf-097e30a71420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:49.489301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b90b2-8f1f-4ad5-baaf-097e30a71420 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.489587 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b90b2-8f1f-4ad5-baaf-097e30a71420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.489890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b90b2-8f1f-4ad5-baaf-097e30a71420 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.490534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b90b2-8f1f-4ad5-baaf-097e30a71420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:49.492454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b90b2-8f1f-4ad5-baaf-097e30a71420 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:49.514374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 979/1971] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:44:49 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:49.518654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236e390-cf6c-4f94-baa8-8804f016f52a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:49.529736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236e390-cf6c-4f94-baa8-8804f016f52a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:49.534402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25f403-299f-4402-9121-0828abf1b655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:49.535715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25f403-299f-4402-9121-0828abf1b655 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.536315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25f403-299f-4402-9121-0828abf1b655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.536635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25f403-299f-4402-9121-0828abf1b655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.537290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25f403-299f-4402-9121-0828abf1b655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:49.538891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25f403-299f-4402-9121-0828abf1b655 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:49.549724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 993/1972] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:49.554620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b90b2-8f1f-4ad5-baaf-097e30a71420 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:49.564544 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b90b2-8f1f-4ad5-baaf-097e30a71420 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:49.569900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbe3aa-d3ae-4a8d-ba4d-7bf816173b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:49.570562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe3aa-d3ae-4a8d-ba4d-7bf816173b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.570848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe3aa-d3ae-4a8d-ba4d-7bf816173b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.571125 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe3aa-d3ae-4a8d-ba4d-7bf816173b66 admin admin] PATH_INFO: `/v3/projects/52fb62dade994233b5e2e23701f32149/users/6ba2cc48836d4b05a797965e16724d10/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.587181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbe3aa-d3ae-4a8d-ba4d-7bf816173b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52fb62dade994233b5e2e23701f32149, user_id=6ba2cc48836d4b05a797965e16724d10, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:49.590077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbe3aa-d3ae-4a8d-ba4d-7bf816173b66 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:49.603811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 980/1973] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:49 2022] PUT /identity/v3/projects/52fb62dade994233b5e2e23701f32149/users/6ba2cc48836d4b05a797965e16724d10/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:49.607810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25f403-299f-4402-9121-0828abf1b655 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:49.618643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25f403-299f-4402-9121-0828abf1b655 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:49.637994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:49.677389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8fff7-3d2c-48ba-a0cb-0e7eaf54f727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:49.678032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8fff7-3d2c-48ba-a0cb-0e7eaf54f727 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.678313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8fff7-3d2c-48ba-a0cb-0e7eaf54f727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.678583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8fff7-3d2c-48ba-a0cb-0e7eaf54f727 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.679227 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8fff7-3d2c-48ba-a0cb-0e7eaf54f727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:49.680950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8fff7-3d2c-48ba-a0cb-0e7eaf54f727 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:49.691399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 994/1974] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:49.695561 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbe3aa-d3ae-4a8d-ba4d-7bf816173b66 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:49.701147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:49.707457 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbe3aa-d3ae-4a8d-ba4d-7bf816173b66 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:49.712257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a7d97-b61c-4f0d-9073-cc6725440671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:49.712926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a7d97-b61c-4f0d-9073-cc6725440671 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.713209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a7d97-b61c-4f0d-9073-cc6725440671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.713489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a7d97-b61c-4f0d-9073-cc6725440671 admin admin] PATH_INFO: `/v3/projects/52fb62dade994233b5e2e23701f32149/users/6ba2cc48836d4b05a797965e16724d10/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.715378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a7d97-b61c-4f0d-9073-cc6725440671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52fb62dade994233b5e2e23701f32149, user_id=6ba2cc48836d4b05a797965e16724d10, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:49.718010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a7d97-b61c-4f0d-9073-cc6725440671 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:49.728467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 981/1975] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:49 2022] PUT /identity/v3/projects/52fb62dade994233b5e2e23701f32149/users/6ba2cc48836d4b05a797965e16724d10/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:49.739296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b37c7-a2fe-4936-8d6d-aa243e1f6505 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.739532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b37c7-a2fe-4936-8d6d-aa243e1f6505 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.739811 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b37c7-a2fe-4936-8d6d-aa243e1f6505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.766677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f46b37c7-a2fe-4936-8d6d-aa243e1f6505 None None] MFA Rules not processed for user `6ba2cc48836d4b05a797965e16724d10`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:49.767876 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f46b37c7-a2fe-4936-8d6d-aa243e1f6505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:49.838648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 995/1976] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:49.850354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a7d97-b61c-4f0d-9073-cc6725440671 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:49.852311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0b0a7d97-b61c-4f0d-9073-cc6725440671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:49.910878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a7d97-b61c-4f0d-9073-cc6725440671 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:49.914570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844b590-c56b-435e-aa70-582d9b9cb3d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:49.915177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844b590-c56b-435e-aa70-582d9b9cb3d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:49.915467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844b590-c56b-435e-aa70-582d9b9cb3d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:49.915762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844b590-c56b-435e-aa70-582d9b9cb3d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:49.924591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844b590-c56b-435e-aa70-582d9b9cb3d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:49.926097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844b590-c56b-435e-aa70-582d9b9cb3d5 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:49.935434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 982/1977] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:50.345679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b37c7-a2fe-4936-8d6d-aa243e1f6505 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:50.415235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b37c7-a2fe-4936-8d6d-aa243e1f6505 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:50.419964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:50.420660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:50.420947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:50.421251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] PATH_INFO: `/v3/users/52ba88535aba4163b12f87aa0fcb838e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:50.422433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ba88535aba4163b12f87aa0fcb838e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:50.424420 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:50.477337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ba88535aba4163b12f87aa0fcb838e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:50.478803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:50.487594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ba88535aba4163b12f87aa0fcb838e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:50.488412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ba88535aba4163b12f87aa0fcb838e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:50.489072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ba88535aba4163b12f87aa0fcb838e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:50.497815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ba88535aba4163b12f87aa0fcb838e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:50.518451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 996/1978] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:44:50 2022] DELETE /identity/v3/users/52ba88535aba4163b12f87aa0fcb838e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:50.523459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8967-cbd8-4178-9ff4-64817c8e4a11 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:50.523751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8967-cbd8-4178-9ff4-64817c8e4a11 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:50.524031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8967-cbd8-4178-9ff4-64817c8e4a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:50.543972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a14e8967-cbd8-4178-9ff4-64817c8e4a11 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:50.545226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a14e8967-cbd8-4178-9ff4-64817c8e4a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:50.614656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 983/1979] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:50.621046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:50.622500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:50.690026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1114b56-78eb-4bd1-bcfc-95eb647b8805 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:50.694090 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc94f84-a2f9-43b3-931b-e9f775f5f53a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:50.694695 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc94f84-a2f9-43b3-931b-e9f775f5f53a service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:50.694975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc94f84-a2f9-43b3-931b-e9f775f5f53a service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:50.695252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc94f84-a2f9-43b3-931b-e9f775f5f53a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:50.704596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc94f84-a2f9-43b3-931b-e9f775f5f53a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:50.706028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc94f84-a2f9-43b3-931b-e9f775f5f53a service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:50.716593 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 997/1980] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:50.995824 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e8967-cbd8-4178-9ff4-64817c8e4a11 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:50.997438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a14e8967-cbd8-4178-9ff4-64817c8e4a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:51.010771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e8967-cbd8-4178-9ff4-64817c8e4a11 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:51.015328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:51.015976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:51.016264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:51.016565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] PATH_INFO: `/v3/projects/ee994e59862847aabde2531555b48251` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:51.017727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee994e59862847aabde2531555b48251)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:51.019755 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:51.066987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee994e59862847aabde2531555b48251', 'request_id': 'req-8f226379-39f2-4551-843e-dcb9c2334170'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.072342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee994e59862847aabde2531555b48251', 'request_id': 'req-8f226379-39f2-4551-843e-dcb9c2334170'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.078738 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee994e59862847aabde2531555b48251', 'request_id': 'req-8f226379-39f2-4551-843e-dcb9c2334170'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.090712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] The token cache is being invalidate because project ee994e59862847aabde2531555b48251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:44:51.091010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.091629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.093475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 984/1981] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:44:50 2022] DELETE /identity/v3/projects/ee994e59862847aabde2531555b48251 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:51.097331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc94f84-a2f9-43b3-931b-e9f775f5f53a service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:51.107890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc94f84-a2f9-43b3-931b-e9f775f5f53a service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:51.163220 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcfeb8-07bb-4c30-985d-7fc11cfc1b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:51.163843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcfeb8-07bb-4c30-985d-7fc11cfc1b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:51.164126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcfeb8-07bb-4c30-985d-7fc11cfc1b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:51.164400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcfeb8-07bb-4c30-985d-7fc11cfc1b28 admin admin] PATH_INFO: `/v3/users/6ba2cc48836d4b05a797965e16724d10` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:51.165543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcfeb8-07bb-4c30-985d-7fc11cfc1b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ba2cc48836d4b05a797965e16724d10)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:51.167456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcfeb8-07bb-4c30-985d-7fc11cfc1b28 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:51.211193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cdfcfeb8-07bb-4c30-985d-7fc11cfc1b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ba2cc48836d4b05a797965e16724d10'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.217993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cdfcfeb8-07bb-4c30-985d-7fc11cfc1b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ba2cc48836d4b05a797965e16724d10'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.218649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cdfcfeb8-07bb-4c30-985d-7fc11cfc1b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ba2cc48836d4b05a797965e16724d10'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.219260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cdfcfeb8-07bb-4c30-985d-7fc11cfc1b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ba2cc48836d4b05a797965e16724d10'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.226531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cdfcfeb8-07bb-4c30-985d-7fc11cfc1b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ba2cc48836d4b05a797965e16724d10'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.231827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:51.246274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 998/1982] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:44:51 2022] DELETE /identity/v3/users/6ba2cc48836d4b05a797965e16724d10 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:51.739429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:51.740965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:51.754663 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f226379-39f2-4551-843e-dcb9c2334170 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:51.768507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:51.822755 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:51.823425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:51.823710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:51.823989 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] PATH_INFO: `/v3/projects/52fb62dade994233b5e2e23701f32149` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:51.825193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52fb62dade994233b5e2e23701f32149)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:51.827142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:51.888872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52fb62dade994233b5e2e23701f32149', 'request_id': 'req-f024eefd-33c0-4d29-9fae-edb3759bd716'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.888872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52fb62dade994233b5e2e23701f32149', 'request_id': 'req-f024eefd-33c0-4d29-9fae-edb3759bd716'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.892347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52fb62dade994233b5e2e23701f32149', 'request_id': 'req-f024eefd-33c0-4d29-9fae-edb3759bd716'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.905248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] The token cache is being invalidate because project 52fb62dade994233b5e2e23701f32149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:44:51.905537 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.906157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f024eefd-33c0-4d29-9fae-edb3759bd716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:44:51.907637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 985/1983] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:44:51 2022] DELETE /identity/v3/projects/52fb62dade994233b5e2e23701f32149 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:51.912817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226df772-54f9-467c-b830-9da67b6d40c5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:51.913301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226df772-54f9-467c-b830-9da67b6d40c5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:51.913398 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226df772-54f9-467c-b830-9da67b6d40c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:51.933129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-226df772-54f9-467c-b830-9da67b6d40c5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:51.934338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-226df772-54f9-467c-b830-9da67b6d40c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:51.995339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 999/1984] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:52.004631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196c79a-63c7-4c8f-9970-01f6498bde43 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.005159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196c79a-63c7-4c8f-9970-01f6498bde43 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.005343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196c79a-63c7-4c8f-9970-01f6498bde43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.025586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7196c79a-63c7-4c8f-9970-01f6498bde43 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:52.026813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7196c79a-63c7-4c8f-9970-01f6498bde43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:52.034631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 986/1985] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:52.039781 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226df772-54f9-467c-b830-9da67b6d40c5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.049408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226df772-54f9-467c-b830-9da67b6d40c5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.054887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5672a-0de5-4e5e-ab73-879602f02992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.055515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5672a-0de5-4e5e-ab73-879602f02992 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.055795 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5672a-0de5-4e5e-ab73-879602f02992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.056065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5672a-0de5-4e5e-ab73-879602f02992 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.056747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5672a-0de5-4e5e-ab73-879602f02992 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:44:52.057051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5672a-0de5-4e5e-ab73-879602f02992 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.058672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5672a-0de5-4e5e-ab73-879602f02992 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.070661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1000/1986] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:44:52.074721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196c79a-63c7-4c8f-9970-01f6498bde43 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.083961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196c79a-63c7-4c8f-9970-01f6498bde43 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.088632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7dd60-d91c-4f46-a4fe-c6a5cba97f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.089228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7dd60-d91c-4f46-a4fe-c6a5cba97f57 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.089510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7dd60-d91c-4f46-a4fe-c6a5cba97f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.089786 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7dd60-d91c-4f46-a4fe-c6a5cba97f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.090422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7dd60-d91c-4f46-a4fe-c6a5cba97f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.092324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7dd60-d91c-4f46-a4fe-c6a5cba97f57 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.119599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 987/1987] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:44:52 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:52.123808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5672a-0de5-4e5e-ab73-879602f02992 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.133918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5672a-0de5-4e5e-ab73-879602f02992 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.193889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c26bc0-1ad4-4855-87c6-715e57ce51c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.194496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c26bc0-1ad4-4855-87c6-715e57ce51c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.194783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c26bc0-1ad4-4855-87c6-715e57ce51c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.195056 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c26bc0-1ad4-4855-87c6-715e57ce51c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.195696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c26bc0-1ad4-4855-87c6-715e57ce51c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.197658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c26bc0-1ad4-4855-87c6-715e57ce51c6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.217856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1001/1988] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:44:52 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:44:52.221912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7dd60-d91c-4f46-a4fe-c6a5cba97f57 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.232636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7dd60-d91c-4f46-a4fe-c6a5cba97f57 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.237268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072b158-64e1-4b67-bd28-2eb844926b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.237872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072b158-64e1-4b67-bd28-2eb844926b74 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.238207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072b158-64e1-4b67-bd28-2eb844926b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.238721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072b158-64e1-4b67-bd28-2eb844926b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.239392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072b158-64e1-4b67-bd28-2eb844926b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.241022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072b158-64e1-4b67-bd28-2eb844926b74 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.250906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 988/1989] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:52.254927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c26bc0-1ad4-4855-87c6-715e57ce51c6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.264751 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c26bc0-1ad4-4855-87c6-715e57ce51c6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.269921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f00926-e578-452c-a60e-f6c0210b125f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.270550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f00926-e578-452c-a60e-f6c0210b125f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.270834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f00926-e578-452c-a60e-f6c0210b125f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.271106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f00926-e578-452c-a60e-f6c0210b125f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.271742 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f00926-e578-452c-a60e-f6c0210b125f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.273365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f00926-e578-452c-a60e-f6c0210b125f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.283785 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1002/1990] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:52.288661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072b158-64e1-4b67-bd28-2eb844926b74 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.298712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072b158-64e1-4b67-bd28-2eb844926b74 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.303220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8900d-81c6-4871-bb6b-138e55fd15a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.303862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8900d-81c6-4871-bb6b-138e55fd15a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.304142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8900d-81c6-4871-bb6b-138e55fd15a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.304414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8900d-81c6-4871-bb6b-138e55fd15a7 admin admin] PATH_INFO: `/v3/projects/126d0ecef1764934b686fbd488f49c3d/users/2092386c942e40a984a31756c18c6efc/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.318982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8900d-81c6-4871-bb6b-138e55fd15a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=126d0ecef1764934b686fbd488f49c3d, user_id=2092386c942e40a984a31756c18c6efc, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.321658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8900d-81c6-4871-bb6b-138e55fd15a7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.334634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 989/1991] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:52 2022] PUT /identity/v3/projects/126d0ecef1764934b686fbd488f49c3d/users/2092386c942e40a984a31756c18c6efc/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:52.345010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaecf3-3b9b-4424-b8ac-68d3822f6923 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.345297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaecf3-3b9b-4424-b8ac-68d3822f6923 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.345575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaecf3-3b9b-4424-b8ac-68d3822f6923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.373260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-30eaecf3-3b9b-4424-b8ac-68d3822f6923 None None] MFA Rules not processed for user `2092386c942e40a984a31756c18c6efc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:52.374466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-30eaecf3-3b9b-4424-b8ac-68d3822f6923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:52.417417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:52.441829 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1003/1992] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:52.446452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8900d-81c6-4871-bb6b-138e55fd15a7 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.492136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:52.511484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8900d-81c6-4871-bb6b-138e55fd15a7 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.516163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795ac48-0927-4bb4-ab08-759d18ca9ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.516817 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795ac48-0927-4bb4-ab08-759d18ca9ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.517105 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795ac48-0927-4bb4-ab08-759d18ca9ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.517383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795ac48-0927-4bb4-ab08-759d18ca9ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.518026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795ac48-0927-4bb4-ab08-759d18ca9ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.520611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795ac48-0927-4bb4-ab08-759d18ca9ea7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.546863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 990/1993] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:44:52 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:52.551881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eaecf3-3b9b-4424-b8ac-68d3822f6923 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.562687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eaecf3-3b9b-4424-b8ac-68d3822f6923 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.619663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c61d79-bba3-46c9-96ce-b2bcdb30a152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.620268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c61d79-bba3-46c9-96ce-b2bcdb30a152 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.620666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c61d79-bba3-46c9-96ce-b2bcdb30a152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.620955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c61d79-bba3-46c9-96ce-b2bcdb30a152 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.621579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c61d79-bba3-46c9-96ce-b2bcdb30a152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.623501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c61d79-bba3-46c9-96ce-b2bcdb30a152 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.645217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1004/1994] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:44:52 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:52.649228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795ac48-0927-4bb4-ab08-759d18ca9ea7 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.659366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795ac48-0927-4bb4-ab08-759d18ca9ea7 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.664424 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0248d04-057c-4783-a4d1-ed480ea48193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.665044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0248d04-057c-4783-a4d1-ed480ea48193 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.665327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0248d04-057c-4783-a4d1-ed480ea48193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.665603 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0248d04-057c-4783-a4d1-ed480ea48193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.666243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0248d04-057c-4783-a4d1-ed480ea48193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.667833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0248d04-057c-4783-a4d1-ed480ea48193 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.679183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 991/1995] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:52.683182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c61d79-bba3-46c9-96ce-b2bcdb30a152 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.693802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c61d79-bba3-46c9-96ce-b2bcdb30a152 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.698689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d416a05-4fbc-4cff-964e-795e85ba23b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.699292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d416a05-4fbc-4cff-964e-795e85ba23b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.699573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d416a05-4fbc-4cff-964e-795e85ba23b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.699845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d416a05-4fbc-4cff-964e-795e85ba23b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.700548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d416a05-4fbc-4cff-964e-795e85ba23b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.702175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d416a05-4fbc-4cff-964e-795e85ba23b9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.712359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1005/1996] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:52.716591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0248d04-057c-4783-a4d1-ed480ea48193 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.726760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0248d04-057c-4783-a4d1-ed480ea48193 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.731565 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605d6bb-d604-4d4f-9b5f-f842f852db65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.732245 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605d6bb-d604-4d4f-9b5f-f842f852db65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.732687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605d6bb-d604-4d4f-9b5f-f842f852db65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.732972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605d6bb-d604-4d4f-9b5f-f842f852db65 admin admin] PATH_INFO: `/v3/projects/d00575ef9b5240a49007618dd9898026/users/df0b0763a6514fd0ba309cbfb0b0e9c0/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.748161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605d6bb-d604-4d4f-9b5f-f842f852db65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00575ef9b5240a49007618dd9898026, user_id=df0b0763a6514fd0ba309cbfb0b0e9c0, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.750915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605d6bb-d604-4d4f-9b5f-f842f852db65 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.767789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 992/1997] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:52 2022] PUT /identity/v3/projects/d00575ef9b5240a49007618dd9898026/users/df0b0763a6514fd0ba309cbfb0b0e9c0/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:44:52.778354 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3e963-f713-4016-b606-5ed2a97ea59f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.778609 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3e963-f713-4016-b606-5ed2a97ea59f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.779017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3e963-f713-4016-b606-5ed2a97ea59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.810505 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-96c3e963-f713-4016-b606-5ed2a97ea59f None None] MFA Rules not processed for user `df0b0763a6514fd0ba309cbfb0b0e9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:52.811729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-96c3e963-f713-4016-b606-5ed2a97ea59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:52.881099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1006/1998] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:52.885862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605d6bb-d604-4d4f-9b5f-f842f852db65 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:52.948453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605d6bb-d604-4d4f-9b5f-f842f852db65 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:52.953217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbb70b-cf4d-4010-adc7-7f4c796ef3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:52.953836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbb70b-cf4d-4010-adc7-7f4c796ef3b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:52.954123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbb70b-cf4d-4010-adc7-7f4c796ef3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:52.954404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbb70b-cf4d-4010-adc7-7f4c796ef3b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:52.955054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbb70b-cf4d-4010-adc7-7f4c796ef3b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:52.957610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbb70b-cf4d-4010-adc7-7f4c796ef3b4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:52.992701 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 993/1999] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:44:52 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:52.996072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3e963-f713-4016-b606-5ed2a97ea59f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:53.011570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3e963-f713-4016-b606-5ed2a97ea59f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:53.068533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:53.079073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135617e-a194-4d7e-b5cd-996b4a57337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:53.079680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135617e-a194-4d7e-b5cd-996b4a57337d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:53.079960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135617e-a194-4d7e-b5cd-996b4a57337d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:53.080230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135617e-a194-4d7e-b5cd-996b4a57337d admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:53.080886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135617e-a194-4d7e-b5cd-996b4a57337d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:53.083345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135617e-a194-4d7e-b5cd-996b4a57337d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:53.105134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1007/2000] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:44:52 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:44:53.109072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbb70b-cf4d-4010-adc7-7f4c796ef3b4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:53.120054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbb70b-cf4d-4010-adc7-7f4c796ef3b4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:53.124725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e469b38-bcba-4c64-b69f-6da9b24f1b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:53.125331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e469b38-bcba-4c64-b69f-6da9b24f1b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:53.125618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e469b38-bcba-4c64-b69f-6da9b24f1b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:53.125900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e469b38-bcba-4c64-b69f-6da9b24f1b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:53.126546 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e469b38-bcba-4c64-b69f-6da9b24f1b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:53.128216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e469b38-bcba-4c64-b69f-6da9b24f1b6b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:53.138887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 994/2001] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:53.143083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135617e-a194-4d7e-b5cd-996b4a57337d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:53.153612 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135617e-a194-4d7e-b5cd-996b4a57337d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:53.158277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7164be8-74f2-4da5-863d-3d3305fb49c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:53.158916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7164be8-74f2-4da5-863d-3d3305fb49c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:53.159200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7164be8-74f2-4da5-863d-3d3305fb49c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:53.159475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7164be8-74f2-4da5-863d-3d3305fb49c8 admin admin] PATH_INFO: `/v3/projects/8b1f1c0a39ff4516ad4620d8bb386192/users/2277c1c4f4b540c6b09dba70ed7c1b39/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:53.174522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7164be8-74f2-4da5-863d-3d3305fb49c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b1f1c0a39ff4516ad4620d8bb386192, user_id=2277c1c4f4b540c6b09dba70ed7c1b39, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:53.177410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7164be8-74f2-4da5-863d-3d3305fb49c8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:53.191930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1008/2002] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:53 2022] PUT /identity/v3/projects/8b1f1c0a39ff4516ad4620d8bb386192/users/2277c1c4f4b540c6b09dba70ed7c1b39/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:53.195892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e469b38-bcba-4c64-b69f-6da9b24f1b6b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:53.205785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e469b38-bcba-4c64-b69f-6da9b24f1b6b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:53.262473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedba03-b9cb-4753-8a51-714a2fd3d65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:53.263089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedba03-b9cb-4753-8a51-714a2fd3d65b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:53.263371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedba03-b9cb-4753-8a51-714a2fd3d65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:53.263648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedba03-b9cb-4753-8a51-714a2fd3d65b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:53.264299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedba03-b9cb-4753-8a51-714a2fd3d65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:53.265940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedba03-b9cb-4753-8a51-714a2fd3d65b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:53.276815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 995/2003] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:44:53.282203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7164be8-74f2-4da5-863d-3d3305fb49c8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:53.293037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7164be8-74f2-4da5-863d-3d3305fb49c8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:53.297542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89984ae9-f2f1-49a8-aed5-19774a817b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:53.298180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89984ae9-f2f1-49a8-aed5-19774a817b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:53.298458 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89984ae9-f2f1-49a8-aed5-19774a817b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:53.298729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89984ae9-f2f1-49a8-aed5-19774a817b2c admin admin] PATH_INFO: `/v3/projects/8b1f1c0a39ff4516ad4620d8bb386192/users/2277c1c4f4b540c6b09dba70ed7c1b39/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:53.301240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89984ae9-f2f1-49a8-aed5-19774a817b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b1f1c0a39ff4516ad4620d8bb386192, user_id=2277c1c4f4b540c6b09dba70ed7c1b39, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:53.303853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89984ae9-f2f1-49a8-aed5-19774a817b2c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:53.313999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1009/2004] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:44:53 2022] PUT /identity/v3/projects/8b1f1c0a39ff4516ad4620d8bb386192/users/2277c1c4f4b540c6b09dba70ed7c1b39/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:44:53.325088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973bdb3-d371-4f8c-b090-cf7bf879c0bc None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:53.325386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973bdb3-d371-4f8c-b090-cf7bf879c0bc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:53.325669 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973bdb3-d371-4f8c-b090-cf7bf879c0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:53.332978 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:53.356002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6973bdb3-d371-4f8c-b090-cf7bf879c0bc None None] MFA Rules not processed for user `2277c1c4f4b540c6b09dba70ed7c1b39`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:44:53.357364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6973bdb3-d371-4f8c-b090-cf7bf879c0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:53.425140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 996/2005] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:44:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:44:53.436813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89984ae9-f2f1-49a8-aed5-19774a817b2c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:53.438520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-89984ae9-f2f1-49a8-aed5-19774a817b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:53.495694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89984ae9-f2f1-49a8-aed5-19774a817b2c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:53.499309 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e552316-b213-4766-9729-1615de2f976d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:53.499903 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e552316-b213-4766-9729-1615de2f976d service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:53.500182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e552316-b213-4766-9729-1615de2f976d service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:53.500459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e552316-b213-4766-9729-1615de2f976d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:53.509182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e552316-b213-4766-9729-1615de2f976d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:53.510599 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e552316-b213-4766-9729-1615de2f976d service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:53.520125 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1010/2006] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:44:54.032713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973bdb3-d371-4f8c-b090-cf7bf879c0bc None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:54.034098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6973bdb3-d371-4f8c-b090-cf7bf879c0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:54.046714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973bdb3-d371-4f8c-b090-cf7bf879c0bc None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:54.050416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e49655-bcc4-419a-b30e-8e4383149f49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:54.051580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e49655-bcc4-419a-b30e-8e4383149f49 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:54.051856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e49655-bcc4-419a-b30e-8e4383149f49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:54.052133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e49655-bcc4-419a-b30e-8e4383149f49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:54.120962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e49655-bcc4-419a-b30e-8e4383149f49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:54.122426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e49655-bcc4-419a-b30e-8e4383149f49 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:54.132445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 997/2007] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:01.411785 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e552316-b213-4766-9729-1615de2f976d service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:01.420184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e552316-b213-4766-9729-1615de2f976d service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:01.423842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bb902-2740-4e2c-a4ad-fd9631e43e64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:01.424436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bb902-2740-4e2c-a4ad-fd9631e43e64 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:01.424754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bb902-2740-4e2c-a4ad-fd9631e43e64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:01.425026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bb902-2740-4e2c-a4ad-fd9631e43e64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:01.482666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bb902-2740-4e2c-a4ad-fd9631e43e64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:01.484088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bb902-2740-4e2c-a4ad-fd9631e43e64 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:01.494007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1011/2008] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:04.127676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=61 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:08.275608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e49655-bcc4-419a-b30e-8e4383149f49 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:08.343488 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e49655-bcc4-419a-b30e-8e4383149f49 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:08.348601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:08.349249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:08.349528 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:08.349803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] PATH_INFO: `/v3/users/2092386c942e40a984a31756c18c6efc` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:08.350946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2092386c942e40a984a31756c18c6efc)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:08.352915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:08.410254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2092386c942e40a984a31756c18c6efc'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:08.410992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2092386c942e40a984a31756c18c6efc'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:08.418987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2092386c942e40a984a31756c18c6efc'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:08.430526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2092386c942e40a984a31756c18c6efc'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:08.431380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2092386c942e40a984a31756c18c6efc'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:08.452067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 998/2009] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:08 2022] DELETE /identity/v3/users/2092386c942e40a984a31756c18c6efc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:08.457120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc75cec-9eb4-4a23-858b-f7fd473c182b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:08.459588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc75cec-9eb4-4a23-858b-f7fd473c182b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:08.459882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc75cec-9eb4-4a23-858b-f7fd473c182b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:08.480388 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4dc75cec-9eb4-4a23-858b-f7fd473c182b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:08.481752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4dc75cec-9eb4-4a23-858b-f7fd473c182b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:08.535021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:08.560563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1012/2010] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:08.571463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:08.573293 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:08.651374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36892fd1-820d-4e6c-91df-8d481bbd4b24 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:08.655125 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fda56-4765-4dc8-8376-43f65d97d2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:08.655740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fda56-4765-4dc8-8376-43f65d97d2eb service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:08.656028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fda56-4765-4dc8-8376-43f65d97d2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:08.656307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fda56-4765-4dc8-8376-43f65d97d2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:08.666278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fda56-4765-4dc8-8376-43f65d97d2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:08.667722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fda56-4765-4dc8-8376-43f65d97d2eb service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:08.685119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 999/2011] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:09.100134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc75cec-9eb4-4a23-858b-f7fd473c182b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:09.101590 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4dc75cec-9eb4-4a23-858b-f7fd473c182b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:09.114544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc75cec-9eb4-4a23-858b-f7fd473c182b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:09.120051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:09.122436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:09.122436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:09.122436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] PATH_INFO: `/v3/projects/126d0ecef1764934b686fbd488f49c3d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:09.122436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] RBAC: Authorizing `identity:delete_project(project_id=126d0ecef1764934b686fbd488f49c3d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:09.123993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:09.176196 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '126d0ecef1764934b686fbd488f49c3d', 'request_id': 'req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:09.182657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '126d0ecef1764934b686fbd488f49c3d', 'request_id': 'req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:09.189365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '126d0ecef1764934b686fbd488f49c3d', 'request_id': 'req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:09.201693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] The token cache is being invalidate because project 126d0ecef1764934b686fbd488f49c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:09.201986 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:09.202599 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:09.204085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1013/2012] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:09 2022] DELETE /identity/v3/projects/126d0ecef1764934b686fbd488f49c3d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:09.208735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fda56-4765-4dc8-8376-43f65d97d2eb service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:09.225956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fda56-4765-4dc8-8376-43f65d97d2eb service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:09.248465 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:09.295500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:09.296175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:09.296436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:09.296705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] PATH_INFO: `/v3/users/df0b0763a6514fd0ba309cbfb0b0e9c0` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:09.297855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0b0763a6514fd0ba309cbfb0b0e9c0)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:09.299981 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:09.358671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0b0763a6514fd0ba309cbfb0b0e9c0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:09.359577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0b0763a6514fd0ba309cbfb0b0e9c0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:09.368345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0b0763a6514fd0ba309cbfb0b0e9c0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:09.381605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0b0763a6514fd0ba309cbfb0b0e9c0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:09.382347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0b0763a6514fd0ba309cbfb0b0e9c0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:09.403880 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1000/2013] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:09 2022] DELETE /identity/v3/users/df0b0763a6514fd0ba309cbfb0b0e9c0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:09.893169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:09.895166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:09.931657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a2e74-7e4f-496b-b640-4dd8ceb7836b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:10.014504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:10.015314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:10.015697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:10.016078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] PATH_INFO: `/v3/projects/d00575ef9b5240a49007618dd9898026` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:10.017386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d00575ef9b5240a49007618dd9898026)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:10.019675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:10.076658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd00575ef9b5240a49007618dd9898026', 'request_id': 'req-5975a70a-80da-425d-a069-18a09b42c433'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.086203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd00575ef9b5240a49007618dd9898026', 'request_id': 'req-5975a70a-80da-425d-a069-18a09b42c433'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.092161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd00575ef9b5240a49007618dd9898026', 'request_id': 'req-5975a70a-80da-425d-a069-18a09b42c433'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.103522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:10.109041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] The token cache is being invalidate because project d00575ef9b5240a49007618dd9898026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:10.109941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.110574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.113624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1014/2014] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:09 2022] DELETE /identity/v3/projects/d00575ef9b5240a49007618dd9898026 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:10.118175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:10.119583 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:10.134438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d4295-d3fb-406d-8a2a-41bb848647e6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:10.211213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34699dcf-3e0b-4e96-948e-43a6036b6729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:10.211982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34699dcf-3e0b-4e96-948e-43a6036b6729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:10.212417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34699dcf-3e0b-4e96-948e-43a6036b6729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:10.212844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34699dcf-3e0b-4e96-948e-43a6036b6729 admin admin] PATH_INFO: `/v3/users/2277c1c4f4b540c6b09dba70ed7c1b39` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:10.214071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34699dcf-3e0b-4e96-948e-43a6036b6729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2277c1c4f4b540c6b09dba70ed7c1b39)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:10.216210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34699dcf-3e0b-4e96-948e-43a6036b6729 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:10.268158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:10.272454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-34699dcf-3e0b-4e96-948e-43a6036b6729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2277c1c4f4b540c6b09dba70ed7c1b39'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.274209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-34699dcf-3e0b-4e96-948e-43a6036b6729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2277c1c4f4b540c6b09dba70ed7c1b39'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.284221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-34699dcf-3e0b-4e96-948e-43a6036b6729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2277c1c4f4b540c6b09dba70ed7c1b39'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.294077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-34699dcf-3e0b-4e96-948e-43a6036b6729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2277c1c4f4b540c6b09dba70ed7c1b39'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.295363 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-34699dcf-3e0b-4e96-948e-43a6036b6729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2277c1c4f4b540c6b09dba70ed7c1b39'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.318868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1001/2015] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:10 2022] DELETE /identity/v3/users/2277c1c4f4b540c6b09dba70ed7c1b39 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:10.618153 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:10.620763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:10.634913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975a70a-80da-425d-a069-18a09b42c433 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:10.702312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:10.702951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:10.703238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:10.703592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] PATH_INFO: `/v3/projects/8b1f1c0a39ff4516ad4620d8bb386192` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:10.704820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b1f1c0a39ff4516ad4620d8bb386192)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:10.707554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:10.761686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b1f1c0a39ff4516ad4620d8bb386192', 'request_id': 'req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.768065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b1f1c0a39ff4516ad4620d8bb386192', 'request_id': 'req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.774292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b1f1c0a39ff4516ad4620d8bb386192', 'request_id': 'req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.789574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] The token cache is being invalidate because project 8b1f1c0a39ff4516ad4620d8bb386192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:10.790082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.790823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90b4cc06-e45a-46e0-8e8b-6d876fa23c6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:10.796568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1015/2016] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:10 2022] DELETE /identity/v3/projects/8b1f1c0a39ff4516ad4620d8bb386192 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:10.800744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d5a08-756a-4dd3-8cfd-9e806ec71643 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:10.801350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d5a08-756a-4dd3-8cfd-9e806ec71643 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:10.802180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d5a08-756a-4dd3-8cfd-9e806ec71643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:10.823693 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-651d5a08-756a-4dd3-8cfd-9e806ec71643 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:10.824969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-651d5a08-756a-4dd3-8cfd-9e806ec71643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:10.903212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1002/2017] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:10.911643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674211e-41c6-4923-bd61-dc5475229a50 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:10.912713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674211e-41c6-4923-bd61-dc5475229a50 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:10.913171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674211e-41c6-4923-bd61-dc5475229a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:10.944009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0674211e-41c6-4923-bd61-dc5475229a50 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:10.945269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0674211e-41c6-4923-bd61-dc5475229a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:10.955204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1016/2018] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:10.960724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d5a08-756a-4dd3-8cfd-9e806ec71643 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:10.973725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d5a08-756a-4dd3-8cfd-9e806ec71643 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:10.981098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e159b-2407-46fe-b98d-8c759a7783f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:10.982217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e159b-2407-46fe-b98d-8c759a7783f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:10.982500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e159b-2407-46fe-b98d-8c759a7783f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:10.982779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e159b-2407-46fe-b98d-8c759a7783f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:10.983582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e159b-2407-46fe-b98d-8c759a7783f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:45:10.984340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e159b-2407-46fe-b98d-8c759a7783f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:10.986125 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e159b-2407-46fe-b98d-8c759a7783f8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:11.009501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1003/2019] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:45:11.013599 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674211e-41c6-4923-bd61-dc5475229a50 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:11.024669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674211e-41c6-4923-bd61-dc5475229a50 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:11.031857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b477117-83da-445a-971d-2cd9ab56a528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:11.032655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b477117-83da-445a-971d-2cd9ab56a528 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:11.033012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b477117-83da-445a-971d-2cd9ab56a528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:11.033290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b477117-83da-445a-971d-2cd9ab56a528 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:11.033937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b477117-83da-445a-971d-2cd9ab56a528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:11.037054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b477117-83da-445a-971d-2cd9ab56a528 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:11.072207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1017/2020] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:11 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:11.073759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e159b-2407-46fe-b98d-8c759a7783f8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:11.087723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e159b-2407-46fe-b98d-8c759a7783f8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:11.193202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67124a8-9793-4fa5-8c01-88266c1c8b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:11.195011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67124a8-9793-4fa5-8c01-88266c1c8b29 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:11.195463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67124a8-9793-4fa5-8c01-88266c1c8b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:11.196036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67124a8-9793-4fa5-8c01-88266c1c8b29 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:11.197123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67124a8-9793-4fa5-8c01-88266c1c8b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:11.202207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67124a8-9793-4fa5-8c01-88266c1c8b29 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:11.235620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1004/2021] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:11 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:11.243680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b477117-83da-445a-971d-2cd9ab56a528 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:11.258516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b477117-83da-445a-971d-2cd9ab56a528 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:11.265216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33967ada-b56a-4ea2-b7d9-40b4272dcccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:11.265384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33967ada-b56a-4ea2-b7d9-40b4272dcccc admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:11.265634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33967ada-b56a-4ea2-b7d9-40b4272dcccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:11.266014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33967ada-b56a-4ea2-b7d9-40b4272dcccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:11.266656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33967ada-b56a-4ea2-b7d9-40b4272dcccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:11.268436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33967ada-b56a-4ea2-b7d9-40b4272dcccc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:11.286305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1018/2022] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:45:11.290012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67124a8-9793-4fa5-8c01-88266c1c8b29 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:11.312308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67124a8-9793-4fa5-8c01-88266c1c8b29 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:11.322610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f70c3-2d5d-437a-a8a0-6547d6005681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:11.323261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f70c3-2d5d-437a-a8a0-6547d6005681 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:11.323553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f70c3-2d5d-437a-a8a0-6547d6005681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:11.323838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f70c3-2d5d-437a-a8a0-6547d6005681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:11.324547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f70c3-2d5d-437a-a8a0-6547d6005681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:11.326563 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f70c3-2d5d-437a-a8a0-6547d6005681 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:11.340451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1005/2023] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:11.345208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33967ada-b56a-4ea2-b7d9-40b4272dcccc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:11.363938 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33967ada-b56a-4ea2-b7d9-40b4272dcccc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:11.370776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a539e97-9d46-43d5-b53d-1ac8f3741cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:11.370776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a539e97-9d46-43d5-b53d-1ac8f3741cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:11.370776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a539e97-9d46-43d5-b53d-1ac8f3741cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:11.370776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a539e97-9d46-43d5-b53d-1ac8f3741cb6 admin admin] PATH_INFO: `/v3/projects/503bc7c970504dfd8079deaaaecf749c/users/65f9abdf8e254512bb266e571f14f1ac/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:11.396550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a539e97-9d46-43d5-b53d-1ac8f3741cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=503bc7c970504dfd8079deaaaecf749c, user_id=65f9abdf8e254512bb266e571f14f1ac, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:11.399351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a539e97-9d46-43d5-b53d-1ac8f3741cb6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:11.416538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1019/2024] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:11 2022] PUT /identity/v3/projects/503bc7c970504dfd8079deaaaecf749c/users/65f9abdf8e254512bb266e571f14f1ac/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:11.429174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b906a-5e2b-43fb-ab19-6829518cc2e7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:11.429464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b906a-5e2b-43fb-ab19-6829518cc2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:11.429741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b906a-5e2b-43fb-ab19-6829518cc2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:11.471137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ab7b906a-5e2b-43fb-ab19-6829518cc2e7 None None] MFA Rules not processed for user `65f9abdf8e254512bb266e571f14f1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:11.473581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ab7b906a-5e2b-43fb-ab19-6829518cc2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:11.490529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:11.554707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1006/2025] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:11.559454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a539e97-9d46-43d5-b53d-1ac8f3741cb6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:11.596119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:11.648315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a539e97-9d46-43d5-b53d-1ac8f3741cb6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:11.653176 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84299ba-8f36-4c56-bd32-93f18b7842b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:11.653784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84299ba-8f36-4c56-bd32-93f18b7842b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:11.654073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84299ba-8f36-4c56-bd32-93f18b7842b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:11.654346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84299ba-8f36-4c56-bd32-93f18b7842b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:11.654988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84299ba-8f36-4c56-bd32-93f18b7842b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:11.656960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84299ba-8f36-4c56-bd32-93f18b7842b4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:11.707268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1020/2026] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:11 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:11.711658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b906a-5e2b-43fb-ab19-6829518cc2e7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:11.725947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b906a-5e2b-43fb-ab19-6829518cc2e7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:11.786270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c667c5-7987-45c5-a541-23d436e1eb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:11.786921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c667c5-7987-45c5-a541-23d436e1eb5b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:11.787215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c667c5-7987-45c5-a541-23d436e1eb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:11.787517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c667c5-7987-45c5-a541-23d436e1eb5b admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:11.788167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c667c5-7987-45c5-a541-23d436e1eb5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:11.790208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c667c5-7987-45c5-a541-23d436e1eb5b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:11.818598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1007/2027] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:11 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:11.820077 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84299ba-8f36-4c56-bd32-93f18b7842b4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:11.830688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84299ba-8f36-4c56-bd32-93f18b7842b4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:11.836158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4060296-5cfa-42da-b464-37bfb34d14dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:11.836794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4060296-5cfa-42da-b464-37bfb34d14dd admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:11.837081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4060296-5cfa-42da-b464-37bfb34d14dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:11.837485 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4060296-5cfa-42da-b464-37bfb34d14dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:11.838142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4060296-5cfa-42da-b464-37bfb34d14dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:11.839779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4060296-5cfa-42da-b464-37bfb34d14dd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:11.850761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1021/2028] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:11.855037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c667c5-7987-45c5-a541-23d436e1eb5b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:11.865657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c667c5-7987-45c5-a541-23d436e1eb5b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:11.870477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7256b-7dd9-4b93-a205-45cb22f94c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:11.871137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7256b-7dd9-4b93-a205-45cb22f94c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:11.871491 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7256b-7dd9-4b93-a205-45cb22f94c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:11.871684 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7256b-7dd9-4b93-a205-45cb22f94c22 admin admin] PATH_INFO: `/v3/projects/646e5ebbdbce464d94bfb714ac79b3c8/users/91d10f5306df486683751c80ac7f9e6d/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:11.887973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7256b-7dd9-4b93-a205-45cb22f94c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646e5ebbdbce464d94bfb714ac79b3c8, user_id=91d10f5306df486683751c80ac7f9e6d, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:11.890876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7256b-7dd9-4b93-a205-45cb22f94c22 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:11.909340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1008/2029] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:11 2022] PUT /identity/v3/projects/646e5ebbdbce464d94bfb714ac79b3c8/users/91d10f5306df486683751c80ac7f9e6d/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:11.913478 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4060296-5cfa-42da-b464-37bfb34d14dd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:11.924588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4060296-5cfa-42da-b464-37bfb34d14dd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:11.989709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7256b-7dd9-4b93-a205-45cb22f94c22 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:11.991917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3ae7256b-7dd9-4b93-a205-45cb22f94c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:11.998890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa126fd3-9aa4-4f86-8f05-093ca9c8b0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:11.999695 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa126fd3-9aa4-4f86-8f05-093ca9c8b0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.000042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa126fd3-9aa4-4f86-8f05-093ca9c8b0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.000804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa126fd3-9aa4-4f86-8f05-093ca9c8b0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.001656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa126fd3-9aa4-4f86-8f05-093ca9c8b0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:12.009074 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7256b-7dd9-4b93-a205-45cb22f94c22 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:12.009839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa126fd3-9aa4-4f86-8f05-093ca9c8b0b2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:12.014757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:12.024042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.024042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.024042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] PATH_INFO: `/v3/users/0d27f92c992749cfa5502babb56cb5a4` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.025368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1022/2030] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:12.027117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d27f92c992749cfa5502babb56cb5a4)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:12.029483 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:12.031224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa126fd3-9aa4-4f86-8f05-093ca9c8b0b2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:12.042661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa126fd3-9aa4-4f86-8f05-093ca9c8b0b2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:12.048240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795c764-505a-4f38-9a42-1feb26351e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:12.048937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795c764-505a-4f38-9a42-1feb26351e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.049221 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795c764-505a-4f38-9a42-1feb26351e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.049503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795c764-505a-4f38-9a42-1feb26351e31 admin admin] PATH_INFO: `/v3/projects/646e5ebbdbce464d94bfb714ac79b3c8/users/91d10f5306df486683751c80ac7f9e6d/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.052115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795c764-505a-4f38-9a42-1feb26351e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646e5ebbdbce464d94bfb714ac79b3c8, user_id=91d10f5306df486683751c80ac7f9e6d, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:12.054803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795c764-505a-4f38-9a42-1feb26351e31 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:12.072530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1023/2031] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:12 2022] PUT /identity/v3/projects/646e5ebbdbce464d94bfb714ac79b3c8/users/91d10f5306df486683751c80ac7f9e6d/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:12.080888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14510c-c991-4250-b02f-7a4ea62f257e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.081298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14510c-c991-4250-b02f-7a4ea62f257e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.081817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14510c-c991-4250-b02f-7a4ea62f257e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.102844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d27f92c992749cfa5502babb56cb5a4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:12.103696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d27f92c992749cfa5502babb56cb5a4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:12.112009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3e14510c-c991-4250-b02f-7a4ea62f257e None None] MFA Rules not processed for user `91d10f5306df486683751c80ac7f9e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:12.112205 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d27f92c992749cfa5502babb56cb5a4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:12.114067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3e14510c-c991-4250-b02f-7a4ea62f257e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:12.120529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d27f92c992749cfa5502babb56cb5a4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:12.121197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d27f92c992749cfa5502babb56cb5a4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:12.131463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:12.140074 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1009/2032] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:11 2022] DELETE /identity/v3/users/0d27f92c992749cfa5502babb56cb5a4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:12.186029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1024/2033] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:45:12.198922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:12.200430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:12.264743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06f207-99cf-40e7-8528-508dbe913ff3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:12.268608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041ef3c-2554-44a9-a9bb-b9497c26abdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:12.269769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ef3c-2554-44a9-a9bb-b9497c26abdf service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.270067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ef3c-2554-44a9-a9bb-b9497c26abdf service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.270411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ef3c-2554-44a9-a9bb-b9497c26abdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.333271 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14510c-c991-4250-b02f-7a4ea62f257e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:12.335292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3e14510c-c991-4250-b02f-7a4ea62f257e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:12.344804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041ef3c-2554-44a9-a9bb-b9497c26abdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:12.353038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:12.359159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041ef3c-2554-44a9-a9bb-b9497c26abdf service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:12.372412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1010/2034] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:12.440259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14510c-c991-4250-b02f-7a4ea62f257e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:12.502451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:12.503776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.504459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.504459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] PATH_INFO: `/v3/projects/11a0149745754ac59b97e489312dd3ab` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.505686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=11a0149745754ac59b97e489312dd3ab)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:12.507715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:12.558635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11a0149745754ac59b97e489312dd3ab', 'request_id': 'req-64d391f0-34f2-4df0-a5cb-edf4f50219fd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:12.564944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11a0149745754ac59b97e489312dd3ab', 'request_id': 'req-64d391f0-34f2-4df0-a5cb-edf4f50219fd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:12.570759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11a0149745754ac59b97e489312dd3ab', 'request_id': 'req-64d391f0-34f2-4df0-a5cb-edf4f50219fd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:12.587429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] The token cache is being invalidate because project 11a0149745754ac59b97e489312dd3ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:12.587721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:12.588365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-64d391f0-34f2-4df0-a5cb-edf4f50219fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:12.589872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1025/2035] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:12 2022] DELETE /identity/v3/projects/11a0149745754ac59b97e489312dd3ab => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:12.594586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094b216-8068-4350-93c6-a06c73c4db16 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.594927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094b216-8068-4350-93c6-a06c73c4db16 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.595208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094b216-8068-4350-93c6-a06c73c4db16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.618647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0094b216-8068-4350-93c6-a06c73c4db16 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:12.619991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0094b216-8068-4350-93c6-a06c73c4db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:12.682342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1011/2036] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:12.690309 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081fb85-ced5-41c3-852c-590869f30d1d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.690929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081fb85-ced5-41c3-852c-590869f30d1d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.691856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081fb85-ced5-41c3-852c-590869f30d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.713145 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8081fb85-ced5-41c3-852c-590869f30d1d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:12.714433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8081fb85-ced5-41c3-852c-590869f30d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:12.724389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1026/2037] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:12.728477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094b216-8068-4350-93c6-a06c73c4db16 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:12.740040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094b216-8068-4350-93c6-a06c73c4db16 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:12.745048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba4f1d-f9a0-49e4-a37c-b65ebdab5008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:12.745652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba4f1d-f9a0-49e4-a37c-b65ebdab5008 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.745939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba4f1d-f9a0-49e4-a37c-b65ebdab5008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.746325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba4f1d-f9a0-49e4-a37c-b65ebdab5008 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.746865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba4f1d-f9a0-49e4-a37c-b65ebdab5008 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:45:12.747177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba4f1d-f9a0-49e4-a37c-b65ebdab5008 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:12.748813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba4f1d-f9a0-49e4-a37c-b65ebdab5008 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:12.761593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1012/2038] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:45:12.765740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081fb85-ced5-41c3-852c-590869f30d1d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:12.775860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081fb85-ced5-41c3-852c-590869f30d1d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:12.780627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cf003-75cc-4794-b104-9b3b16a7e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:12.781249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cf003-75cc-4794-b104-9b3b16a7e7cf admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.781533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cf003-75cc-4794-b104-9b3b16a7e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.781805 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cf003-75cc-4794-b104-9b3b16a7e7cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.782446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cf003-75cc-4794-b104-9b3b16a7e7cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:12.784910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cf003-75cc-4794-b104-9b3b16a7e7cf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:12.818178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1027/2039] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:12 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:12.822193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba4f1d-f9a0-49e4-a37c-b65ebdab5008 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:12.832436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba4f1d-f9a0-49e4-a37c-b65ebdab5008 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:12.863326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:12.898618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021db48-b3d9-4c24-9cb2-bd38ea69829e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:12.899260 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021db48-b3d9-4c24-9cb2-bd38ea69829e admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.899550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021db48-b3d9-4c24-9cb2-bd38ea69829e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.899833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021db48-b3d9-4c24-9cb2-bd38ea69829e admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.900503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021db48-b3d9-4c24-9cb2-bd38ea69829e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:12.902657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021db48-b3d9-4c24-9cb2-bd38ea69829e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:12.926235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1013/2040] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:12 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:12.930074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cf003-75cc-4794-b104-9b3b16a7e7cf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:12.941889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cf003-75cc-4794-b104-9b3b16a7e7cf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:12.947422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326f023-9ea3-4947-aefc-86ac0ca4f0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:12.948123 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326f023-9ea3-4947-aefc-86ac0ca4f0bb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.948477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326f023-9ea3-4947-aefc-86ac0ca4f0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.948857 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326f023-9ea3-4947-aefc-86ac0ca4f0bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.949576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326f023-9ea3-4947-aefc-86ac0ca4f0bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:12.951198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326f023-9ea3-4947-aefc-86ac0ca4f0bb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:12.962301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1028/2041] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:12.966315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021db48-b3d9-4c24-9cb2-bd38ea69829e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:12.977034 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021db48-b3d9-4c24-9cb2-bd38ea69829e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:12.984831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85b88c-264c-4448-ae8d-05cf359988da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:12.985441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85b88c-264c-4448-ae8d-05cf359988da admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:12.985710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85b88c-264c-4448-ae8d-05cf359988da admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:12.985983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85b88c-264c-4448-ae8d-05cf359988da admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:12.986625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85b88c-264c-4448-ae8d-05cf359988da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:12.988388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85b88c-264c-4448-ae8d-05cf359988da admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.000338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326f023-9ea3-4947-aefc-86ac0ca4f0bb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.002421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9326f023-9ea3-4947-aefc-86ac0ca4f0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.004613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1014/2042] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:13.007644 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85b88c-264c-4448-ae8d-05cf359988da admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.015020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:13.019883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85b88c-264c-4448-ae8d-05cf359988da admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.020718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326f023-9ea3-4947-aefc-86ac0ca4f0bb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.025988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d177370-6758-4389-8f86-cf1da6a865cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.026798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d177370-6758-4389-8f86-cf1da6a865cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.027087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d177370-6758-4389-8f86-cf1da6a865cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.027370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d177370-6758-4389-8f86-cf1da6a865cb admin admin] PATH_INFO: `/v3/projects/34115e05045744aea98a88679eb4f3b2/users/3a7aa6def512412ca70bebc294bc29ce/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.027528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.028219 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.028595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.028940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] PATH_INFO: `/v3/users/65f9abdf8e254512bb266e571f14f1ac` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.030140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f9abdf8e254512bb266e571f14f1ac)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.032158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.044105 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d177370-6758-4389-8f86-cf1da6a865cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=34115e05045744aea98a88679eb4f3b2, user_id=3a7aa6def512412ca70bebc294bc29ce, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.046821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d177370-6758-4389-8f86-cf1da6a865cb admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.061632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1015/2043] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:13 2022] PUT /identity/v3/projects/34115e05045744aea98a88679eb4f3b2/users/3a7aa6def512412ca70bebc294bc29ce/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:13.073407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef896c2-25a8-4e86-abad-254631da9e1d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.073703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef896c2-25a8-4e86-abad-254631da9e1d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.074136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef896c2-25a8-4e86-abad-254631da9e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.085890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f9abdf8e254512bb266e571f14f1ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:13.092995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f9abdf8e254512bb266e571f14f1ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:13.093732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f9abdf8e254512bb266e571f14f1ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:13.094347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f9abdf8e254512bb266e571f14f1ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:13.105830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1ef896c2-25a8-4e86-abad-254631da9e1d None None] MFA Rules not processed for user `3a7aa6def512412ca70bebc294bc29ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:13.106055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f9abdf8e254512bb266e571f14f1ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:13.107100 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1ef896c2-25a8-4e86-abad-254631da9e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.130863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1029/2044] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:12 2022] DELETE /identity/v3/users/65f9abdf8e254512bb266e571f14f1ac => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:13.131762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.133416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.182283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1016/2045] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:45:13.183802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7431a-3d83-4ec9-ac46-52d707a626ed None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.184166 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7431a-3d83-4ec9-ac46-52d707a626ed None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.184921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7431a-3d83-4ec9-ac46-52d707a626ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.207859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-33e7431a-3d83-4ec9-ac46-52d707a626ed None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:13.208713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-33e7431a-3d83-4ec9-ac46-52d707a626ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.218065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1017/2046] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:13.218912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e7431a-3d83-4ec9-ac46-52d707a626ed None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.220217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-33e7431a-3d83-4ec9-ac46-52d707a626ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.222544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca258e86-1948-47b0-a243-ced810a9cae1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.228567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.229418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.229719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.230756 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] PATH_INFO: `/v3/users/14c2fa3118c74e128f3307ab1efcb0b4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.232063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c2fa3118c74e128f3307ab1efcb0b4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.234109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.235103 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e7431a-3d83-4ec9-ac46-52d707a626ed None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.239995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcf520-a873-4af7-abde-d7404df26059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.240685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcf520-a873-4af7-abde-d7404df26059 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.240912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcf520-a873-4af7-abde-d7404df26059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.241194 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcf520-a873-4af7-abde-d7404df26059 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.241829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcf520-a873-4af7-abde-d7404df26059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.243735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcf520-a873-4af7-abde-d7404df26059 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.272621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1018/2047] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:13 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:13.273727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcf520-a873-4af7-abde-d7404df26059 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.276913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f1dcf520-a873-4af7-abde-d7404df26059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.286057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c2fa3118c74e128f3307ab1efcb0b4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:13.293308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c2fa3118c74e128f3307ab1efcb0b4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:13.294237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c2fa3118c74e128f3307ab1efcb0b4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:13.294877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c2fa3118c74e128f3307ab1efcb0b4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:13.302934 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c2fa3118c74e128f3307ab1efcb0b4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:13.315333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:13.324748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1030/2048] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:13 2022] DELETE /identity/v3/users/14c2fa3118c74e128f3307ab1efcb0b4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:13.325696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.328565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.333424 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:13.338564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcf520-a873-4af7-abde-d7404df26059 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.389222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6753d-90cb-402d-afe8-ca82b698c93b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.389955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6753d-90cb-402d-afe8-ca82b698c93b service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.390212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6753d-90cb-402d-afe8-ca82b698c93b service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.390499 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6753d-90cb-402d-afe8-ca82b698c93b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.393294 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0fb6753d-90cb-402d-afe8-ca82b698c93b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.395967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20624217-db67-41a5-95ae-b6ccf8671281 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.400688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d2748-097e-48b7-9d90-232765185746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.401288 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2748-097e-48b7-9d90-232765185746 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.401573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2748-097e-48b7-9d90-232765185746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.401848 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2748-097e-48b7-9d90-232765185746 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.402550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d2748-097e-48b7-9d90-232765185746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.404473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d2748-097e-48b7-9d90-232765185746 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.406242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6753d-90cb-402d-afe8-ca82b698c93b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.407689 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6753d-90cb-402d-afe8-ca82b698c93b service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.418113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1019/2049] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:13.419628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eab3e-60ec-4d77-839d-db44dec51593 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.420078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eab3e-60ec-4d77-839d-db44dec51593 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.420342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eab3e-60ec-4d77-839d-db44dec51593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.427674 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1031/2050] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:13 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:13.431909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d2748-097e-48b7-9d90-232765185746 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.440355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-269eab3e-60ec-4d77-839d-db44dec51593 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:13.442263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-269eab3e-60ec-4d77-839d-db44dec51593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.444820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d2748-097e-48b7-9d90-232765185746 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.449702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97064955-31af-4306-bf5e-635036784a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.450334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97064955-31af-4306-bf5e-635036784a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.450548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1020/2051] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:13.450909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97064955-31af-4306-bf5e-635036784a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.451193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97064955-31af-4306-bf5e-635036784a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.451959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97064955-31af-4306-bf5e-635036784a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.453600 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97064955-31af-4306-bf5e-635036784a3a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.461855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eab3e-60ec-4d77-839d-db44dec51593 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.463509 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-269eab3e-60ec-4d77-839d-db44dec51593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.466144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1032/2052] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:13.469977 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97064955-31af-4306-bf5e-635036784a3a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.476760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eab3e-60ec-4d77-839d-db44dec51593 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.481083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87e0f9-b6f8-4940-9c84-62f1a498aba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.481738 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87e0f9-b6f8-4940-9c84-62f1a498aba5 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.481959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87e0f9-b6f8-4940-9c84-62f1a498aba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.482069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97064955-31af-4306-bf5e-635036784a3a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.482187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87e0f9-b6f8-4940-9c84-62f1a498aba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.487587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c41e38-cbfb-43cd-a811-69edb299f042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.488514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c41e38-cbfb-43cd-a811-69edb299f042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.488935 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c41e38-cbfb-43cd-a811-69edb299f042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.489282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c41e38-cbfb-43cd-a811-69edb299f042 admin admin] PATH_INFO: `/v3/projects/6ffd5b1dd04b44f6a4bcf7614bdfad6d/users/65157bf04400490893f82580dc11e667/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.492193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87e0f9-b6f8-4940-9c84-62f1a498aba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.494035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87e0f9-b6f8-4940-9c84-62f1a498aba5 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.508055 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1021/2053] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:45:13.513376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c41e38-cbfb-43cd-a811-69edb299f042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffd5b1dd04b44f6a4bcf7614bdfad6d, user_id=65157bf04400490893f82580dc11e667, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.516184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c41e38-cbfb-43cd-a811-69edb299f042 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.533527 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1033/2054] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:13 2022] PUT /identity/v3/projects/6ffd5b1dd04b44f6a4bcf7614bdfad6d/users/65157bf04400490893f82580dc11e667/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:13.538581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87e0f9-b6f8-4940-9c84-62f1a498aba5 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.550883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87e0f9-b6f8-4940-9c84-62f1a498aba5 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.622274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa816840-a1ee-4756-908e-b66844ddd908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.623564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa816840-a1ee-4756-908e-b66844ddd908 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.623996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa816840-a1ee-4756-908e-b66844ddd908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.625094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa816840-a1ee-4756-908e-b66844ddd908 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.625864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa816840-a1ee-4756-908e-b66844ddd908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.628406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa816840-a1ee-4756-908e-b66844ddd908 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.645641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1022/2055] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:13.649507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c41e38-cbfb-43cd-a811-69edb299f042 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.662616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c41e38-cbfb-43cd-a811-69edb299f042 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.672650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5246b4-2217-452a-91ea-7c19edb8abd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.673465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5246b4-2217-452a-91ea-7c19edb8abd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.673985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5246b4-2217-452a-91ea-7c19edb8abd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.675398 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5246b4-2217-452a-91ea-7c19edb8abd5 admin admin] PATH_INFO: `/v3/projects/6ffd5b1dd04b44f6a4bcf7614bdfad6d/users/65157bf04400490893f82580dc11e667/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.692313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5246b4-2217-452a-91ea-7c19edb8abd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffd5b1dd04b44f6a4bcf7614bdfad6d, user_id=65157bf04400490893f82580dc11e667, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.695596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5246b4-2217-452a-91ea-7c19edb8abd5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.710783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1034/2056] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:13 2022] PUT /identity/v3/projects/6ffd5b1dd04b44f6a4bcf7614bdfad6d/users/65157bf04400490893f82580dc11e667/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:13.721621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edc6b0-2ecc-43de-abf4-9b82f515a7cf None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.721621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edc6b0-2ecc-43de-abf4-9b82f515a7cf None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.722517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edc6b0-2ecc-43de-abf4-9b82f515a7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.752711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b9edc6b0-2ecc-43de-abf4-9b82f515a7cf None None] MFA Rules not processed for user `65157bf04400490893f82580dc11e667`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:13.753836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b9edc6b0-2ecc-43de-abf4-9b82f515a7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.833101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5246b4-2217-452a-91ea-7c19edb8abd5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.834962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ab5246b4-2217-452a-91ea-7c19edb8abd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.886564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1023/2057] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:13.887400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edc6b0-2ecc-43de-abf4-9b82f515a7cf None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:13.889137 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b9edc6b0-2ecc-43de-abf4-9b82f515a7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:13.902390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:13.925424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5246b4-2217-452a-91ea-7c19edb8abd5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.933276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.933997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.934353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.934696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] PATH_INFO: `/v3/projects/503bc7c970504dfd8079deaaaecf749c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.936178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=503bc7c970504dfd8079deaaaecf749c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.938405 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.961401 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edc6b0-2ecc-43de-abf4-9b82f515a7cf None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:13.968832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:13.969487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:13.969801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:13.970053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] PATH_INFO: `/v3/projects/3b4f340d2f194b33bf5bde41e61fb1a5` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:13.971184 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b4f340d2f194b33bf5bde41e61fb1a5)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:13.973263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:13.982257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:13.990590 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '503bc7c970504dfd8079deaaaecf749c', 'request_id': 'req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:13.997847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '503bc7c970504dfd8079deaaaecf749c', 'request_id': 'req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.004622 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '503bc7c970504dfd8079deaaaecf749c', 'request_id': 'req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.017483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] The token cache is being invalidate because project 503bc7c970504dfd8079deaaaecf749c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:14.017778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.018430 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.020039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1035/2058] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:13 2022] DELETE /identity/v3/projects/503bc7c970504dfd8079deaaaecf749c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:14.020846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.022144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.026228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b4f340d2f194b33bf5bde41e61fb1a5', 'request_id': 'req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.032266 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b4f340d2f194b33bf5bde41e61fb1a5', 'request_id': 'req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.039253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b4f340d2f194b33bf5bde41e61fb1a5', 'request_id': 'req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.053372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] The token cache is being invalidate because project 3b4f340d2f194b33bf5bde41e61fb1a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:14.053802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.054560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.056074 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1024/2059] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:13 2022] DELETE /identity/v3/projects/3b4f340d2f194b33bf5bde41e61fb1a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:14.060362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.061686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.082041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f5a9c-ebb0-49bb-b56b-998bb6f7d357 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:14.127734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b55f2-3bd3-4fe2-895c-dc7d373857b9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:14.132724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:14.132921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d3460-be78-4c55-b313-95e9f7732465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:14.133411 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.133512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d3460-be78-4c55-b313-95e9f7732465 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.133700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.133804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d3460-be78-4c55-b313-95e9f7732465 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.133980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] PATH_INFO: `/v3/users/91d10f5306df486683751c80ac7f9e6d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.134084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d3460-be78-4c55-b313-95e9f7732465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.135311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=91d10f5306df486683751c80ac7f9e6d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:14.135865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-681d3460-be78-4c55-b313-95e9f7732465 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.137323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:14.179984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:14.186385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91d10f5306df486683751c80ac7f9e6d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.187489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91d10f5306df486683751c80ac7f9e6d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.196048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91d10f5306df486683751c80ac7f9e6d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.203116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d3460-be78-4c55-b313-95e9f7732465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:14.204767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91d10f5306df486683751c80ac7f9e6d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.204907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d3460-be78-4c55-b313-95e9f7732465 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:14.205883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91d10f5306df486683751c80ac7f9e6d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.215907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1036/2060] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:14.216994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4dbba-6535-41a4-a1d4-6c6bf3b66edf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.217281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4dbba-6535-41a4-a1d4-6c6bf3b66edf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.217565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4dbba-6535-41a4-a1d4-6c6bf3b66edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.226249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1025/2061] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:14 2022] DELETE /identity/v3/users/91d10f5306df486683751c80ac7f9e6d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:14.243987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-00b4dbba-6535-41a4-a1d4-6c6bf3b66edf None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:14.245306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-00b4dbba-6535-41a4-a1d4-6c6bf3b66edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.267915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.271046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.285302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:14.331268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1037/2062] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:14.341669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ddb62-4b0a-426a-b098-f545cd2287e1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.342145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e28ae-b318-404e-b7a1-9060929846fa admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:14.342292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ddb62-4b0a-426a-b098-f545cd2287e1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.342462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ddb62-4b0a-426a-b098-f545cd2287e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.346164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79152e-ec02-4a5b-bf99-69b1dafcdc26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:14.346927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79152e-ec02-4a5b-bf99-69b1dafcdc26 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.347315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79152e-ec02-4a5b-bf99-69b1dafcdc26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.347711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79152e-ec02-4a5b-bf99-69b1dafcdc26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.349913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cd79152e-ec02-4a5b-bf99-69b1dafcdc26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.365031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-347ddb62-4b0a-426a-b098-f545cd2287e1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:14.366237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-347ddb62-4b0a-426a-b098-f545cd2287e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.374529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1038/2063] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:14.379027 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ddb62-4b0a-426a-b098-f545cd2287e1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.388184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ddb62-4b0a-426a-b098-f545cd2287e1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:14.394378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bb0b4-b33c-476a-b1f9-ce78d23f38b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:14.394990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bb0b4-b33c-476a-b1f9-ce78d23f38b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.395271 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bb0b4-b33c-476a-b1f9-ce78d23f38b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.395541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bb0b4-b33c-476a-b1f9-ce78d23f38b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.396182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bb0b4-b33c-476a-b1f9-ce78d23f38b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:45:14.408660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bb0b4-b33c-476a-b1f9-ce78d23f38b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:14.410607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bb0b4-b33c-476a-b1f9-ce78d23f38b5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:14.424523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1039/2064] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:45:14.427385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79152e-ec02-4a5b-bf99-69b1dafcdc26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:14.428891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79152e-ec02-4a5b-bf99-69b1dafcdc26 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:14.429575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bb0b4-b33c-476a-b1f9-ce78d23f38b5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.438493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1026/2065] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:45:14.440233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bb0b4-b33c-476a-b1f9-ce78d23f38b5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:14.445934 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a64ee-b08c-4248-8c18-473532c3463e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:14.446517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a64ee-b08c-4248-8c18-473532c3463e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.446781 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a64ee-b08c-4248-8c18-473532c3463e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.447053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a64ee-b08c-4248-8c18-473532c3463e admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.447692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a64ee-b08c-4248-8c18-473532c3463e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:14.449762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a64ee-b08c-4248-8c18-473532c3463e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:14.478174 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1040/2066] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:14 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:14.483698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79152e-ec02-4a5b-bf99-69b1dafcdc26 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.497652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79152e-ec02-4a5b-bf99-69b1dafcdc26 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:14.546925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a64ee-b08c-4248-8c18-473532c3463e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.548900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f53a64ee-b08c-4248-8c18-473532c3463e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.572106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e42098-2a8c-40d4-9ee6-98e505221296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:14.573376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e42098-2a8c-40d4-9ee6-98e505221296 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.573768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e42098-2a8c-40d4-9ee6-98e505221296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.574432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e42098-2a8c-40d4-9ee6-98e505221296 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.575182 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e42098-2a8c-40d4-9ee6-98e505221296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:14.577384 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e42098-2a8c-40d4-9ee6-98e505221296 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:14.599938 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:14.606055 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1027/2067] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:14 2022] POST /identity/v3/users => generated 296 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:14.610202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e42098-2a8c-40d4-9ee6-98e505221296 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.622638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e42098-2a8c-40d4-9ee6-98e505221296 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:14.624970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a64ee-b08c-4248-8c18-473532c3463e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:14.628044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47037f-253f-4b43-a48a-a4222afb7c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:14.629240 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47037f-253f-4b43-a48a-a4222afb7c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.629527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47037f-253f-4b43-a48a-a4222afb7c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.629796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47037f-253f-4b43-a48a-a4222afb7c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.630440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47037f-253f-4b43-a48a-a4222afb7c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:14.630635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:14.631382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.631532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.631809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] PATH_INFO: `/v3/projects/646e5ebbdbce464d94bfb714ac79b3c8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.632052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47037f-253f-4b43-a48a-a4222afb7c4b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:14.632971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646e5ebbdbce464d94bfb714ac79b3c8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:14.634917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:14.643673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1028/2068] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:14.648326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47037f-253f-4b43-a48a-a4222afb7c4b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.662175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47037f-253f-4b43-a48a-a4222afb7c4b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:14.668428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bbfa5-2d7a-4fd1-8fbb-cf01fa17ab2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:14.669314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bbfa5-2d7a-4fd1-8fbb-cf01fa17ab2b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.669504 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bbfa5-2d7a-4fd1-8fbb-cf01fa17ab2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.669896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bbfa5-2d7a-4fd1-8fbb-cf01fa17ab2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.670576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3bbfa5-2d7a-4fd1-8fbb-cf01fa17ab2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:14.672591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3bbfa5-2d7a-4fd1-8fbb-cf01fa17ab2b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:14.691885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1029/2069] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:45:14.696353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bbfa5-2d7a-4fd1-8fbb-cf01fa17ab2b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.699616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646e5ebbdbce464d94bfb714ac79b3c8', 'request_id': 'req-f24f011a-a4f2-47ef-87d0-36d356aecde9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.707385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646e5ebbdbce464d94bfb714ac79b3c8', 'request_id': 'req-f24f011a-a4f2-47ef-87d0-36d356aecde9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.708921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bbfa5-2d7a-4fd1-8fbb-cf01fa17ab2b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:14.714537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646e5ebbdbce464d94bfb714ac79b3c8', 'request_id': 'req-f24f011a-a4f2-47ef-87d0-36d356aecde9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.729309 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] The token cache is being invalidate because project 646e5ebbdbce464d94bfb714ac79b3c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:14.729604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.730265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f24f011a-a4f2-47ef-87d0-36d356aecde9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:14.735851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1041/2070] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:14 2022] DELETE /identity/v3/projects/646e5ebbdbce464d94bfb714ac79b3c8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:14.753279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a2253-3246-4447-9dfc-9c4640db4f0c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.754030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a2253-3246-4447-9dfc-9c4640db4f0c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.754030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a2253-3246-4447-9dfc-9c4640db4f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.773025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b442511-6954-4f02-930c-1a6d79f44249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:14.773548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b442511-6954-4f02-930c-1a6d79f44249 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.773831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b442511-6954-4f02-930c-1a6d79f44249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.774114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b442511-6954-4f02-930c-1a6d79f44249 admin admin] PATH_INFO: `/v3/projects/3164dc1f6a1047f78dd20c649a10aed4/users/fe2b0a7ecdb0457184969a98c53f1599/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.775075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e28a2253-3246-4447-9dfc-9c4640db4f0c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:14.776389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e28a2253-3246-4447-9dfc-9c4640db4f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.791890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1042/2071] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:14.797328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b442511-6954-4f02-930c-1a6d79f44249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3164dc1f6a1047f78dd20c649a10aed4, user_id=fe2b0a7ecdb0457184969a98c53f1599, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:14.801351 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b442511-6954-4f02-930c-1a6d79f44249 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:14.806719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd320d-623f-4d54-95d8-3c47958bdcd5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.807025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd320d-623f-4d54-95d8-3c47958bdcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.807328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd320d-623f-4d54-95d8-3c47958bdcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.819519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1030/2072] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:14 2022] PUT /identity/v3/projects/3164dc1f6a1047f78dd20c649a10aed4/users/fe2b0a7ecdb0457184969a98c53f1599/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:14.830703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9375be-d819-4fee-9ed1-48207b94d269 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.831379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9375be-d819-4fee-9ed1-48207b94d269 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.831979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9375be-d819-4fee-9ed1-48207b94d269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.853276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-acdd320d-623f-4d54-95d8-3c47958bdcd5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:14.854477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-acdd320d-623f-4d54-95d8-3c47958bdcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.866073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:14.881049 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cd9375be-d819-4fee-9ed1-48207b94d269 None None] MFA Rules not processed for user `fe2b0a7ecdb0457184969a98c53f1599`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:14.882695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cd9375be-d819-4fee-9ed1-48207b94d269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.942224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1043/2073] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:45:14.946277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd320d-623f-4d54-95d8-3c47958bdcd5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.956098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd320d-623f-4d54-95d8-3c47958bdcd5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:14.961613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb60b9-ca7c-4e6d-9c53-c75f5bdd16ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:14.962325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb60b9-ca7c-4e6d-9c53-c75f5bdd16ad admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:14.962535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb60b9-ca7c-4e6d-9c53-c75f5bdd16ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:14.962810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb60b9-ca7c-4e6d-9c53-c75f5bdd16ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:14.963464 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb60b9-ca7c-4e6d-9c53-c75f5bdd16ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:45:14.963880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb60b9-ca7c-4e6d-9c53-c75f5bdd16ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:14.965306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1031/2074] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:14.965526 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb60b9-ca7c-4e6d-9c53-c75f5bdd16ad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:14.977563 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9375be-d819-4fee-9ed1-48207b94d269 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:14.979418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cd9375be-d819-4fee-9ed1-48207b94d269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:14.981317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1044/2075] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:45:14.989155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb60b9-ca7c-4e6d-9c53-c75f5bdd16ad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:15.000792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb60b9-ca7c-4e6d-9c53-c75f5bdd16ad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:15.007219 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591acc4-3a66-4468-b9d7-c4a016dbf736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:15.008053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591acc4-3a66-4468-b9d7-c4a016dbf736 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:15.008391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591acc4-3a66-4468-b9d7-c4a016dbf736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:15.008791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591acc4-3a66-4468-b9d7-c4a016dbf736 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:15.009612 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591acc4-3a66-4468-b9d7-c4a016dbf736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:15.012233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591acc4-3a66-4468-b9d7-c4a016dbf736 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:15.042975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1045/2076] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:14 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:15.046968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591acc4-3a66-4468-b9d7-c4a016dbf736 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:15.058278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9375be-d819-4fee-9ed1-48207b94d269 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:15.062053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:15.114900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda3847-e905-4aa9-b323-620b30137d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:15.115521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3847-e905-4aa9-b323-620b30137d59 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:15.115776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591acc4-3a66-4468-b9d7-c4a016dbf736 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:15.115888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3847-e905-4aa9-b323-620b30137d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:15.116070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3847-e905-4aa9-b323-620b30137d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:15.120464 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6064518-2609-4b48-8631-160c052c1672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:15.121079 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6064518-2609-4b48-8631-160c052c1672 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:15.121361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6064518-2609-4b48-8631-160c052c1672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:15.121715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6064518-2609-4b48-8631-160c052c1672 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:15.122348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6064518-2609-4b48-8631-160c052c1672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:15.124238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6064518-2609-4b48-8631-160c052c1672 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:15.152262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1046/2077] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:15 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:15.156241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6064518-2609-4b48-8631-160c052c1672 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:15.166691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6064518-2609-4b48-8631-160c052c1672 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:15.171509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b3786-6882-4324-96ca-4840566f660e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:15.172103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b3786-6882-4324-96ca-4840566f660e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:15.172384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b3786-6882-4324-96ca-4840566f660e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:15.172698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b3786-6882-4324-96ca-4840566f660e admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:15.173351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b3786-6882-4324-96ca-4840566f660e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:15.174965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b3786-6882-4324-96ca-4840566f660e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:15.180631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda3847-e905-4aa9-b323-620b30137d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:15.182185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda3847-e905-4aa9-b323-620b30137d59 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:15.186116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1047/2078] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:15.190080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b3786-6882-4324-96ca-4840566f660e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:15.192069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1032/2079] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:45:15.200245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b3786-6882-4324-96ca-4840566f660e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:15.204852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df740e-29b4-4ba4-aa75-5a17675c6c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:15.205445 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df740e-29b4-4ba4-aa75-5a17675c6c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:15.205731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df740e-29b4-4ba4-aa75-5a17675c6c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:15.206003 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df740e-29b4-4ba4-aa75-5a17675c6c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:15.207424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df740e-29b4-4ba4-aa75-5a17675c6c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:15.209138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df740e-29b4-4ba4-aa75-5a17675c6c77 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:15.223903 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1048/2080] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:45:15.225264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda3847-e905-4aa9-b323-620b30137d59 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:15.238372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda3847-e905-4aa9-b323-620b30137d59 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:15.243239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60f907-7020-4139-abfc-e44ec8b8b8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:15.243900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60f907-7020-4139-abfc-e44ec8b8b8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:15.244187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60f907-7020-4139-abfc-e44ec8b8b8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:15.244463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60f907-7020-4139-abfc-e44ec8b8b8f4 admin admin] PATH_INFO: `/v3/projects/444aa9c6ee34483587001e470f88cc17/users/00f153678d3547ccbe0bbbe3df93ed9d/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:15.260542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60f907-7020-4139-abfc-e44ec8b8b8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444aa9c6ee34483587001e470f88cc17, user_id=00f153678d3547ccbe0bbbe3df93ed9d, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:15.263234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60f907-7020-4139-abfc-e44ec8b8b8f4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:15.277116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1033/2081] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:15 2022] PUT /identity/v3/projects/444aa9c6ee34483587001e470f88cc17/users/00f153678d3547ccbe0bbbe3df93ed9d/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:15.289459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a811d48-b4b2-4b83-b175-280cd5fd8151 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:15.289738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a811d48-b4b2-4b83-b175-280cd5fd8151 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:15.290008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a811d48-b4b2-4b83-b175-280cd5fd8151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:15.319650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1a811d48-b4b2-4b83-b175-280cd5fd8151 None None] MFA Rules not processed for user `00f153678d3547ccbe0bbbe3df93ed9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:15.321108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a811d48-b4b2-4b83-b175-280cd5fd8151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:15.365323 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:15.394683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1049/2082] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:15.407553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60f907-7020-4139-abfc-e44ec8b8b8f4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:15.409395 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ed60f907-7020-4139-abfc-e44ec8b8b8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:15.478608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60f907-7020-4139-abfc-e44ec8b8b8f4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:15.482924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a815ea-e4aa-4196-a246-559242288a8c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ec048788904bc58245e878a8644091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:15.485151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a815ea-e4aa-4196-a246-559242288a8c service glance] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:15.485603 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a815ea-e4aa-4196-a246-559242288a8c service glance] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:15.485885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a815ea-e4aa-4196-a246-559242288a8c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:15.495860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a815ea-e4aa-4196-a246-559242288a8c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:15.497696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a815ea-e4aa-4196-a246-559242288a8c service glance] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:15.509675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1034/2083] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:17.912749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a811d48-b4b2-4b83-b175-280cd5fd8151 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:17.914207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a811d48-b4b2-4b83-b175-280cd5fd8151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:17.979755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a811d48-b4b2-4b83-b175-280cd5fd8151 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:17.984239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:17.984896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:17.985179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:17.985453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] PATH_INFO: `/v3/users/3a7aa6def512412ca70bebc294bc29ce` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:17.986577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7aa6def512412ca70bebc294bc29ce)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:17.988579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:18.039392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7aa6def512412ca70bebc294bc29ce'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.046958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7aa6def512412ca70bebc294bc29ce'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.047693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7aa6def512412ca70bebc294bc29ce'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.048365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7aa6def512412ca70bebc294bc29ce'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.057296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7aa6def512412ca70bebc294bc29ce'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.075849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1050/2084] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:17 2022] DELETE /identity/v3/users/3a7aa6def512412ca70bebc294bc29ce => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:18.080279 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c536b-913e-4e01-9e57-ad5a36b1529c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:18.080789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c536b-913e-4e01-9e57-ad5a36b1529c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:18.080929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c536b-913e-4e01-9e57-ad5a36b1529c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:18.100085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-738c536b-913e-4e01-9e57-ad5a36b1529c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:18.101255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-738c536b-913e-4e01-9e57-ad5a36b1529c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:18.135460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:18.161922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1035/2085] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:18.171446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:18.172858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:18.231926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cf3af-b567-4a4a-be92-c42d5bec0297 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:18.236056 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420e185-1326-43d7-95c7-848b82837616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:18.236697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420e185-1326-43d7-95c7-848b82837616 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:18.236979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420e185-1326-43d7-95c7-848b82837616 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:18.237252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420e185-1326-43d7-95c7-848b82837616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:18.246873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420e185-1326-43d7-95c7-848b82837616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:18.248287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420e185-1326-43d7-95c7-848b82837616 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:18.253187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:18.259500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1051/2086] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:45:18.531834 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c536b-913e-4e01-9e57-ad5a36b1529c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:18.533938 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-738c536b-913e-4e01-9e57-ad5a36b1529c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:18.551203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c536b-913e-4e01-9e57-ad5a36b1529c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:18.555810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:18.556446 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:18.556761 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:18.557703 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] PATH_INFO: `/v3/projects/34115e05045744aea98a88679eb4f3b2` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:18.559312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34115e05045744aea98a88679eb4f3b2)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:18.561213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:18.607173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34115e05045744aea98a88679eb4f3b2', 'request_id': 'req-bbd73fb6-3062-4cec-962e-678fcdebd750'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.613814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34115e05045744aea98a88679eb4f3b2', 'request_id': 'req-bbd73fb6-3062-4cec-962e-678fcdebd750'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.619892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34115e05045744aea98a88679eb4f3b2', 'request_id': 'req-bbd73fb6-3062-4cec-962e-678fcdebd750'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.632309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] The token cache is being invalidate because project 34115e05045744aea98a88679eb4f3b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:18.632633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.633263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.634742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1036/2087] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:18 2022] DELETE /identity/v3/projects/34115e05045744aea98a88679eb4f3b2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:18.638676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420e185-1326-43d7-95c7-848b82837616 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:18.648298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420e185-1326-43d7-95c7-848b82837616 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:18.709167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f975f-a249-4db9-833d-602ad12b658d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:18.709796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f975f-a249-4db9-833d-602ad12b658d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:18.710077 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f975f-a249-4db9-833d-602ad12b658d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:18.710351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f975f-a249-4db9-833d-602ad12b658d admin admin] PATH_INFO: `/v3/users/65157bf04400490893f82580dc11e667` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:18.711463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f975f-a249-4db9-833d-602ad12b658d admin admin] RBAC: Authorizing `identity:delete_user(user_id=65157bf04400490893f82580dc11e667)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:18.713425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f975f-a249-4db9-833d-602ad12b658d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:18.758940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-234f975f-a249-4db9-833d-602ad12b658d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65157bf04400490893f82580dc11e667'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.765770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-234f975f-a249-4db9-833d-602ad12b658d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65157bf04400490893f82580dc11e667'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.766437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-234f975f-a249-4db9-833d-602ad12b658d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65157bf04400490893f82580dc11e667'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.767116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-234f975f-a249-4db9-833d-602ad12b658d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65157bf04400490893f82580dc11e667'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.774817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-234f975f-a249-4db9-833d-602ad12b658d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65157bf04400490893f82580dc11e667'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:18.793100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1052/2088] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:18 2022] DELETE /identity/v3/users/65157bf04400490893f82580dc11e667 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:19.071955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:19.073534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:19.103673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd73fb6-3062-4cec-962e-678fcdebd750 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:19.164309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:19.164903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:19.165184 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:19.165457 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] PATH_INFO: `/v3/projects/6ffd5b1dd04b44f6a4bcf7614bdfad6d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:19.166572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffd5b1dd04b44f6a4bcf7614bdfad6d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:19.168535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:19.213770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffd5b1dd04b44f6a4bcf7614bdfad6d', 'request_id': 'req-badf08db-12de-4dfe-b0f0-280b488ac2e4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:19.220269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffd5b1dd04b44f6a4bcf7614bdfad6d', 'request_id': 'req-badf08db-12de-4dfe-b0f0-280b488ac2e4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:19.228271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffd5b1dd04b44f6a4bcf7614bdfad6d', 'request_id': 'req-badf08db-12de-4dfe-b0f0-280b488ac2e4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:19.231620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:19.243072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] The token cache is being invalidate because project 6ffd5b1dd04b44f6a4bcf7614bdfad6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:19.243362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:19.243978 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-badf08db-12de-4dfe-b0f0-280b488ac2e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:19.245530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1037/2089] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:19 2022] DELETE /identity/v3/projects/6ffd5b1dd04b44f6a4bcf7614bdfad6d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:19.250864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26787ed2-73f0-482e-ac07-eda42e6ded6c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:19.251333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26787ed2-73f0-482e-ac07-eda42e6ded6c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:19.251457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26787ed2-73f0-482e-ac07-eda42e6ded6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:19.273789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-26787ed2-73f0-482e-ac07-eda42e6ded6c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:19.275001 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-26787ed2-73f0-482e-ac07-eda42e6ded6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:19.336622 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1053/2090] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:19.344150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d807274-415d-4bd5-b82d-45e21420fb3b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:19.344439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d807274-415d-4bd5-b82d-45e21420fb3b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:19.344751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d807274-415d-4bd5-b82d-45e21420fb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:19.364124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1d807274-415d-4bd5-b82d-45e21420fb3b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:19.365356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1d807274-415d-4bd5-b82d-45e21420fb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:19.373720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1038/2091] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:19.378146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26787ed2-73f0-482e-ac07-eda42e6ded6c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:19.389004 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26787ed2-73f0-482e-ac07-eda42e6ded6c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:19.393652 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6c600-9efb-4a0c-865c-24114ca1fc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:19.394928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6c600-9efb-4a0c-865c-24114ca1fc97 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:19.395197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6c600-9efb-4a0c-865c-24114ca1fc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:19.395467 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6c600-9efb-4a0c-865c-24114ca1fc97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:19.396135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6c600-9efb-4a0c-865c-24114ca1fc97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:45:19.396441 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6c600-9efb-4a0c-865c-24114ca1fc97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:19.398053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6c600-9efb-4a0c-865c-24114ca1fc97 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:19.409700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1054/2092] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:45:19.414302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d807274-415d-4bd5-b82d-45e21420fb3b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:19.424081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d807274-415d-4bd5-b82d-45e21420fb3b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:19.429890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68cabf-241c-4913-8b45-49e1881cc68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:19.430505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68cabf-241c-4913-8b45-49e1881cc68b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:19.430789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68cabf-241c-4913-8b45-49e1881cc68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:19.431148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68cabf-241c-4913-8b45-49e1881cc68b admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:19.431795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68cabf-241c-4913-8b45-49e1881cc68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:19.433859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68cabf-241c-4913-8b45-49e1881cc68b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:19.463831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1039/2093] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:19 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:19.467977 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6c600-9efb-4a0c-865c-24114ca1fc97 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:19.477971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6c600-9efb-4a0c-865c-24114ca1fc97 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:19.481546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:19.543617 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f319bb4-2a33-49aa-9e57-a4348e194562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:19.544215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f319bb4-2a33-49aa-9e57-a4348e194562 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:19.544514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f319bb4-2a33-49aa-9e57-a4348e194562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:19.544791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f319bb4-2a33-49aa-9e57-a4348e194562 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:19.545402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f319bb4-2a33-49aa-9e57-a4348e194562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:19.547278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f319bb4-2a33-49aa-9e57-a4348e194562 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:19.572474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1055/2094] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:19 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:19.576582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68cabf-241c-4913-8b45-49e1881cc68b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:19.587847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68cabf-241c-4913-8b45-49e1881cc68b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:19.592539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08178c4-05a0-4d4b-824e-67905729762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:19.593138 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08178c4-05a0-4d4b-824e-67905729762b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:19.593418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08178c4-05a0-4d4b-824e-67905729762b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:19.593692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08178c4-05a0-4d4b-824e-67905729762b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:19.594332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08178c4-05a0-4d4b-824e-67905729762b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:19.595934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08178c4-05a0-4d4b-824e-67905729762b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:19.606109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1040/2095] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:19.610591 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f319bb4-2a33-49aa-9e57-a4348e194562 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:19.623217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f319bb4-2a33-49aa-9e57-a4348e194562 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:19.628022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72626836-5f03-47c1-aede-d2397e72516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:19.629941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72626836-5f03-47c1-aede-d2397e72516e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:19.630232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72626836-5f03-47c1-aede-d2397e72516e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:19.630510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72626836-5f03-47c1-aede-d2397e72516e admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:19.631162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72626836-5f03-47c1-aede-d2397e72516e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:19.632806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72626836-5f03-47c1-aede-d2397e72516e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:19.645294 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1056/2096] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:19.649720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08178c4-05a0-4d4b-824e-67905729762b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:19.659997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08178c4-05a0-4d4b-824e-67905729762b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:19.664694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edee5f-add6-4d7f-a681-79284290debc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:19.665350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edee5f-add6-4d7f-a681-79284290debc admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:19.665635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edee5f-add6-4d7f-a681-79284290debc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:19.665915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edee5f-add6-4d7f-a681-79284290debc admin admin] PATH_INFO: `/v3/projects/9a2951273e46453b9ef89c0f39724127/users/a15e185e8c994b1f8093321093b6e7db/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:19.686682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3edee5f-add6-4d7f-a681-79284290debc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a2951273e46453b9ef89c0f39724127, user_id=a15e185e8c994b1f8093321093b6e7db, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:19.689408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3edee5f-add6-4d7f-a681-79284290debc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:19.704796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1041/2097] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:19 2022] PUT /identity/v3/projects/9a2951273e46453b9ef89c0f39724127/users/a15e185e8c994b1f8093321093b6e7db/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:19.716735 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f0ba8-9c4c-45ba-a317-c79db957931d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:19.717090 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f0ba8-9c4c-45ba-a317-c79db957931d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:19.717310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f0ba8-9c4c-45ba-a317-c79db957931d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:19.752033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-576f0ba8-9c4c-45ba-a317-c79db957931d None None] MFA Rules not processed for user `a15e185e8c994b1f8093321093b6e7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:19.753928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-576f0ba8-9c4c-45ba-a317-c79db957931d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:19.830744 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1057/2098] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:19.835593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edee5f-add6-4d7f-a681-79284290debc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:19.898574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edee5f-add6-4d7f-a681-79284290debc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:19.903284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d090-e0f1-470f-8e64-acdd67dd9d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:19.903888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d090-e0f1-470f-8e64-acdd67dd9d42 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:19.904172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d090-e0f1-470f-8e64-acdd67dd9d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:19.904447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d090-e0f1-470f-8e64-acdd67dd9d42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:19.905119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566d090-e0f1-470f-8e64-acdd67dd9d42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:19.907039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566d090-e0f1-470f-8e64-acdd67dd9d42 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:19.940194 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1042/2099] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:19 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:19.945120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f0ba8-9c4c-45ba-a317-c79db957931d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:19.955812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f0ba8-9c4c-45ba-a317-c79db957931d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:20.012709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624837a2-e8ff-4796-8922-d1db919d2259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:20.013321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624837a2-e8ff-4796-8922-d1db919d2259 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:20.014620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624837a2-e8ff-4796-8922-d1db919d2259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:20.014892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624837a2-e8ff-4796-8922-d1db919d2259 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:20.015523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624837a2-e8ff-4796-8922-d1db919d2259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:20.017543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624837a2-e8ff-4796-8922-d1db919d2259 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:20.038958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1058/2100] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:19 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:20.043011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d090-e0f1-470f-8e64-acdd67dd9d42 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:20.054037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d090-e0f1-470f-8e64-acdd67dd9d42 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:20.058746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d3fcc-23ff-41ca-95d5-f83555a9aa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:20.059348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d3fcc-23ff-41ca-95d5-f83555a9aa75 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:20.059632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d3fcc-23ff-41ca-95d5-f83555a9aa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:20.059909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d3fcc-23ff-41ca-95d5-f83555a9aa75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:20.060593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d3fcc-23ff-41ca-95d5-f83555a9aa75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:20.062208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d3fcc-23ff-41ca-95d5-f83555a9aa75 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:20.074255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1043/2101] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:20.078446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624837a2-e8ff-4796-8922-d1db919d2259 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:20.084951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:20.089065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624837a2-e8ff-4796-8922-d1db919d2259 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:20.094924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc04ee-1b04-496a-9419-0f8f05c65d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:20.095579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc04ee-1b04-496a-9419-0f8f05c65d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:20.095858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc04ee-1b04-496a-9419-0f8f05c65d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:20.096130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc04ee-1b04-496a-9419-0f8f05c65d22 admin admin] PATH_INFO: `/v3/projects/e0a92e01c5b945378a95daab5d658aea/users/918e77ff64254f0fb2a548f6ff362e0c/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:20.111900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc04ee-1b04-496a-9419-0f8f05c65d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a92e01c5b945378a95daab5d658aea, user_id=918e77ff64254f0fb2a548f6ff362e0c, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:20.115265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc04ee-1b04-496a-9419-0f8f05c65d22 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:20.127868 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1059/2102] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:20 2022] PUT /identity/v3/projects/e0a92e01c5b945378a95daab5d658aea/users/918e77ff64254f0fb2a548f6ff362e0c/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:20.131952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d3fcc-23ff-41ca-95d5-f83555a9aa75 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:20.142324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d3fcc-23ff-41ca-95d5-f83555a9aa75 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:20.200576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d663f6-35e3-467f-aaa5-fd56159ed61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:20.201125 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d663f6-35e3-467f-aaa5-fd56159ed61e admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:20.201405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d663f6-35e3-467f-aaa5-fd56159ed61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:20.201676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d663f6-35e3-467f-aaa5-fd56159ed61e admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:20.202319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d663f6-35e3-467f-aaa5-fd56159ed61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:20.203896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d663f6-35e3-467f-aaa5-fd56159ed61e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:20.209068 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:20.216479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1044/2103] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:20.218843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc04ee-1b04-496a-9419-0f8f05c65d22 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:20.230312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc04ee-1b04-496a-9419-0f8f05c65d22 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:20.235387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c18645-b694-43ac-b08a-47120c1e7c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:20.236025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c18645-b694-43ac-b08a-47120c1e7c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:20.236304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c18645-b694-43ac-b08a-47120c1e7c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:20.236598 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c18645-b694-43ac-b08a-47120c1e7c06 admin admin] PATH_INFO: `/v3/projects/e0a92e01c5b945378a95daab5d658aea/users/918e77ff64254f0fb2a548f6ff362e0c/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:20.239166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c18645-b694-43ac-b08a-47120c1e7c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a92e01c5b945378a95daab5d658aea, user_id=918e77ff64254f0fb2a548f6ff362e0c, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:20.241818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c18645-b694-43ac-b08a-47120c1e7c06 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:20.251959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1060/2104] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:20 2022] PUT /identity/v3/projects/e0a92e01c5b945378a95daab5d658aea/users/918e77ff64254f0fb2a548f6ff362e0c/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:20.262228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c7ba6-0d6a-4c94-acde-9b7701b4b93f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:20.262517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c7ba6-0d6a-4c94-acde-9b7701b4b93f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:20.262795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c7ba6-0d6a-4c94-acde-9b7701b4b93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:20.290678 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3a5c7ba6-0d6a-4c94-acde-9b7701b4b93f None None] MFA Rules not processed for user `918e77ff64254f0fb2a548f6ff362e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:20.291892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3a5c7ba6-0d6a-4c94-acde-9b7701b4b93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:20.360598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1045/2105] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:20.372152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c18645-b694-43ac-b08a-47120c1e7c06 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:20.373984 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-91c18645-b694-43ac-b08a-47120c1e7c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:20.434454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c18645-b694-43ac-b08a-47120c1e7c06 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:20.438150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78686-2801-40f2-8662-f8f1a1014701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:20.438751 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78686-2801-40f2-8662-f8f1a1014701 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:20.439035 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78686-2801-40f2-8662-f8f1a1014701 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:20.439330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78686-2801-40f2-8662-f8f1a1014701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:20.449205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c78686-2801-40f2-8662-f8f1a1014701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:20.450698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c78686-2801-40f2-8662-f8f1a1014701 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:20.460960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1061/2106] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:20 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:20.509698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c7ba6-0d6a-4c94-acde-9b7701b4b93f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:20.511116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3a5c7ba6-0d6a-4c94-acde-9b7701b4b93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:20.524109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c7ba6-0d6a-4c94-acde-9b7701b4b93f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:20.527988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80172737-e9d0-4e45-8dc3-b4f483e0e66f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:20.528732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80172737-e9d0-4e45-8dc3-b4f483e0e66f service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:20.529019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80172737-e9d0-4e45-8dc3-b4f483e0e66f service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:20.529434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80172737-e9d0-4e45-8dc3-b4f483e0e66f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:20.591425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80172737-e9d0-4e45-8dc3-b4f483e0e66f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:20.592925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80172737-e9d0-4e45-8dc3-b4f483e0e66f service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:20.602769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1046/2107] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:30.268699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78686-2801-40f2-8662-f8f1a1014701 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:30.358929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78686-2801-40f2-8662-f8f1a1014701 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:30.376299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:30.377113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:30.377404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:30.377686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] PATH_INFO: `/v3/users/a15e185e8c994b1f8093321093b6e7db` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:30.378991 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a15e185e8c994b1f8093321093b6e7db)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:30.385506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:30.470875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a15e185e8c994b1f8093321093b6e7db'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:30.479558 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a15e185e8c994b1f8093321093b6e7db'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:30.480318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a15e185e8c994b1f8093321093b6e7db'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:30.480963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a15e185e8c994b1f8093321093b6e7db'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:30.489356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a15e185e8c994b1f8093321093b6e7db'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:30.499045 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:30.520535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1062/2108] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:30 2022] DELETE /identity/v3/users/a15e185e8c994b1f8093321093b6e7db => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:30.523152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894087b-069f-4b0c-84d3-d5752ee6f2de None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:30.523799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894087b-069f-4b0c-84d3-d5752ee6f2de None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:30.523799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894087b-069f-4b0c-84d3-d5752ee6f2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:30.558246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4894087b-069f-4b0c-84d3-d5752ee6f2de None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:30.562255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4894087b-069f-4b0c-84d3-d5752ee6f2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:30.644449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1047/2109] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:30.654902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:30.656518 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:30.730030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e80a5e-39a6-4ffc-a778-95bbef6681ad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:30.733585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fae64-9836-4926-a781-d7cade8854b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:30.734190 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fae64-9836-4926-a781-d7cade8854b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:30.734468 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fae64-9836-4926-a781-d7cade8854b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:30.734739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fae64-9836-4926-a781-d7cade8854b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:30.744353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fae64-9836-4926-a781-d7cade8854b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:30.745800 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fae64-9836-4926-a781-d7cade8854b0 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:30.755559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1063/2110] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:45:31.036316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894087b-069f-4b0c-84d3-d5752ee6f2de None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:31.038610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4894087b-069f-4b0c-84d3-d5752ee6f2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:31.051252 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894087b-069f-4b0c-84d3-d5752ee6f2de None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:31.057676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:31.058383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:31.058693 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:31.059009 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] PATH_INFO: `/v3/projects/9a2951273e46453b9ef89c0f39724127` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:31.060298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a2951273e46453b9ef89c0f39724127)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:31.069203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:31.092160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:31.118075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a2951273e46453b9ef89c0f39724127', 'request_id': 'req-eb7ac399-aa30-4858-87f0-8d1ba66fb233'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.123539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a2951273e46453b9ef89c0f39724127', 'request_id': 'req-eb7ac399-aa30-4858-87f0-8d1ba66fb233'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.129921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a2951273e46453b9ef89c0f39724127', 'request_id': 'req-eb7ac399-aa30-4858-87f0-8d1ba66fb233'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.142084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] The token cache is being invalidate because project 9a2951273e46453b9ef89c0f39724127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:31.142372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.142984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.144506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1048/2111] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:31 2022] DELETE /identity/v3/projects/9a2951273e46453b9ef89c0f39724127 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:31.149660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fae64-9836-4926-a781-d7cade8854b0 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:31.160794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fae64-9836-4926-a781-d7cade8854b0 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:31.219552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a3983-ff52-48e1-bd95-53ac834be4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:31.220184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a3983-ff52-48e1-bd95-53ac834be4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:31.220463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a3983-ff52-48e1-bd95-53ac834be4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:31.220766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a3983-ff52-48e1-bd95-53ac834be4a5 admin admin] PATH_INFO: `/v3/users/918e77ff64254f0fb2a548f6ff362e0c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:31.221872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a3983-ff52-48e1-bd95-53ac834be4a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=918e77ff64254f0fb2a548f6ff362e0c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:31.223842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a3983-ff52-48e1-bd95-53ac834be4a5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:31.274669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7e9a3983-ff52-48e1-bd95-53ac834be4a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '918e77ff64254f0fb2a548f6ff362e0c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.281566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7e9a3983-ff52-48e1-bd95-53ac834be4a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '918e77ff64254f0fb2a548f6ff362e0c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.282239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7e9a3983-ff52-48e1-bd95-53ac834be4a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '918e77ff64254f0fb2a548f6ff362e0c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.282854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7e9a3983-ff52-48e1-bd95-53ac834be4a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '918e77ff64254f0fb2a548f6ff362e0c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.290473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7e9a3983-ff52-48e1-bd95-53ac834be4a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '918e77ff64254f0fb2a548f6ff362e0c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.307845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1064/2112] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:31 2022] DELETE /identity/v3/users/918e77ff64254f0fb2a548f6ff362e0c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:31.650627 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:31.652250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:31.665586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ac399-aa30-4858-87f0-8d1ba66fb233 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:31.737372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:31.738128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:31.738545 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:31.738944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] PATH_INFO: `/v3/projects/e0a92e01c5b945378a95daab5d658aea` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:31.741226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a92e01c5b945378a95daab5d658aea)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:31.743441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:31.792316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a92e01c5b945378a95daab5d658aea', 'request_id': 'req-e62dfe86-c187-41b1-838c-443ca9474544'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.797790 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a92e01c5b945378a95daab5d658aea', 'request_id': 'req-e62dfe86-c187-41b1-838c-443ca9474544'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.805582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a92e01c5b945378a95daab5d658aea', 'request_id': 'req-e62dfe86-c187-41b1-838c-443ca9474544'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.820968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] The token cache is being invalidate because project e0a92e01c5b945378a95daab5d658aea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:31.821265 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.821894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e62dfe86-c187-41b1-838c-443ca9474544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:31.823386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1049/2113] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:31 2022] DELETE /identity/v3/projects/e0a92e01c5b945378a95daab5d658aea => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:31.828311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01f384-f788-4e79-9ebd-74e6597a3049 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:31.828726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01f384-f788-4e79-9ebd-74e6597a3049 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:31.829168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01f384-f788-4e79-9ebd-74e6597a3049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:31.855752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-de01f384-f788-4e79-9ebd-74e6597a3049 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:31.857088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-de01f384-f788-4e79-9ebd-74e6597a3049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:31.863273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:31.923001 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1065/2114] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:45:31.931793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84cea1-3477-41e1-84c8-8d4791131996 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:31.932081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84cea1-3477-41e1-84c8-8d4791131996 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:31.932358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84cea1-3477-41e1-84c8-8d4791131996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:31.952843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0b84cea1-3477-41e1-84c8-8d4791131996 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:31.954037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0b84cea1-3477-41e1-84c8-8d4791131996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:31.962004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1050/2115] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:31.966338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01f384-f788-4e79-9ebd-74e6597a3049 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:31.977199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01f384-f788-4e79-9ebd-74e6597a3049 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:31.982388 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4d3ca-585d-417b-8801-00cf07912f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:31.983010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4d3ca-585d-417b-8801-00cf07912f5e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:31.983294 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4d3ca-585d-417b-8801-00cf07912f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:31.983568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4d3ca-585d-417b-8801-00cf07912f5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:31.984221 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4d3ca-585d-417b-8801-00cf07912f5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:45:31.984557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4d3ca-585d-417b-8801-00cf07912f5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:31.986197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4d3ca-585d-417b-8801-00cf07912f5e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:31.999640 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1066/2116] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:45:32.004073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84cea1-3477-41e1-84c8-8d4791131996 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.013396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84cea1-3477-41e1-84c8-8d4791131996 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:32.018241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81db32-00be-47ad-94f4-0f2868b20b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:32.018877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81db32-00be-47ad-94f4-0f2868b20b74 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.019165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81db32-00be-47ad-94f4-0f2868b20b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.019447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81db32-00be-47ad-94f4-0f2868b20b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.020089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81db32-00be-47ad-94f4-0f2868b20b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:32.022206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81db32-00be-47ad-94f4-0f2868b20b74 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:32.053422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1051/2117] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:32 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:32.057584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4d3ca-585d-417b-8801-00cf07912f5e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.067571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4d3ca-585d-417b-8801-00cf07912f5e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:32.124446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2cce0-aff4-4750-82c9-ecd680001c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:32.125078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2cce0-aff4-4750-82c9-ecd680001c85 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.125358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2cce0-aff4-4750-82c9-ecd680001c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.125630 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2cce0-aff4-4750-82c9-ecd680001c85 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.126263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2cce0-aff4-4750-82c9-ecd680001c85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:32.128207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2cce0-aff4-4750-82c9-ecd680001c85 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:32.148796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1067/2118] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:32 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:32.152954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81db32-00be-47ad-94f4-0f2868b20b74 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.163646 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81db32-00be-47ad-94f4-0f2868b20b74 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:32.168250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96b956-185d-4a08-9261-b43c398f3346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:32.168886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96b956-185d-4a08-9261-b43c398f3346 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.169168 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96b956-185d-4a08-9261-b43c398f3346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.169443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96b956-185d-4a08-9261-b43c398f3346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.170086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96b956-185d-4a08-9261-b43c398f3346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:32.171688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96b956-185d-4a08-9261-b43c398f3346 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:32.173838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:32.184544 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1052/2119] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:32.188030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2cce0-aff4-4750-82c9-ecd680001c85 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.198649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2cce0-aff4-4750-82c9-ecd680001c85 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:32.203267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f959b8-dcd7-4ba2-a442-9dee4e7593df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:32.203885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f959b8-dcd7-4ba2-a442-9dee4e7593df admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.204164 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f959b8-dcd7-4ba2-a442-9dee4e7593df admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.204439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f959b8-dcd7-4ba2-a442-9dee4e7593df admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.205143 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f959b8-dcd7-4ba2-a442-9dee4e7593df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:32.206748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f959b8-dcd7-4ba2-a442-9dee4e7593df admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:32.218216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1068/2120] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:32.222321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96b956-185d-4a08-9261-b43c398f3346 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.237772 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96b956-185d-4a08-9261-b43c398f3346 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:32.243412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5dbdbe-4f5f-48be-a4d1-166ecfb89681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:32.244100 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dbdbe-4f5f-48be-a4d1-166ecfb89681 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.244386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dbdbe-4f5f-48be-a4d1-166ecfb89681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.244697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dbdbe-4f5f-48be-a4d1-166ecfb89681 admin admin] PATH_INFO: `/v3/projects/e269b10fde2d450798e4d36225a35351/users/413d2d8e754d46feab6931702ba0496f/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.261046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dbdbe-4f5f-48be-a4d1-166ecfb89681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e269b10fde2d450798e4d36225a35351, user_id=413d2d8e754d46feab6931702ba0496f, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:32.263705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dbdbe-4f5f-48be-a4d1-166ecfb89681 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:32.276803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1053/2121] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:32 2022] PUT /identity/v3/projects/e269b10fde2d450798e4d36225a35351/users/413d2d8e754d46feab6931702ba0496f/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:32.288431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444caa9b-b144-4aa7-b171-e87a2488b046 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.288964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444caa9b-b144-4aa7-b171-e87a2488b046 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.289060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444caa9b-b144-4aa7-b171-e87a2488b046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.317397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-444caa9b-b144-4aa7-b171-e87a2488b046 None None] MFA Rules not processed for user `413d2d8e754d46feab6931702ba0496f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:32.318593 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-444caa9b-b144-4aa7-b171-e87a2488b046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:32.382609 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1069/2122] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:32.387297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5dbdbe-4f5f-48be-a4d1-166ecfb89681 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.449775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5dbdbe-4f5f-48be-a4d1-166ecfb89681 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:32.454290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625f8ea-94d0-41fc-ad72-4d312076fb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:32.454888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f8ea-94d0-41fc-ad72-4d312076fb46 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.455167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f8ea-94d0-41fc-ad72-4d312076fb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.455439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f8ea-94d0-41fc-ad72-4d312076fb46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.456061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625f8ea-94d0-41fc-ad72-4d312076fb46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:32.457984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625f8ea-94d0-41fc-ad72-4d312076fb46 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:32.483448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1054/2123] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:32 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:32.487643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444caa9b-b144-4aa7-b171-e87a2488b046 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.499234 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444caa9b-b144-4aa7-b171-e87a2488b046 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:32.502805 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:32.566022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1838832-bed8-4979-9c43-94d630248706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:32.566628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1838832-bed8-4979-9c43-94d630248706 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.566911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1838832-bed8-4979-9c43-94d630248706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.567182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1838832-bed8-4979-9c43-94d630248706 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.568523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1838832-bed8-4979-9c43-94d630248706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:32.570444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1838832-bed8-4979-9c43-94d630248706 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:32.590313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1070/2124] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:32 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:32.594315 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625f8ea-94d0-41fc-ad72-4d312076fb46 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.604804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625f8ea-94d0-41fc-ad72-4d312076fb46 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:32.609397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce7205-3100-45fc-9853-25495e2b560a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:32.610011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce7205-3100-45fc-9853-25495e2b560a admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.610301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce7205-3100-45fc-9853-25495e2b560a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.610574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce7205-3100-45fc-9853-25495e2b560a admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.611211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce7205-3100-45fc-9853-25495e2b560a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:32.612862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce7205-3100-45fc-9853-25495e2b560a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:32.627445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1055/2125] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:32.628943 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1838832-bed8-4979-9c43-94d630248706 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.642901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1838832-bed8-4979-9c43-94d630248706 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:32.649188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dda89-cd92-4c17-aa88-81159454ace8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:32.649985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dda89-cd92-4c17-aa88-81159454ace8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.650450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dda89-cd92-4c17-aa88-81159454ace8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.650798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dda89-cd92-4c17-aa88-81159454ace8 admin admin] PATH_INFO: `/v3/projects/48cd9b81ce48475aad2714e4b67a6496/users/22461a0d12b94583892357cb2be49866/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.667790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9dda89-cd92-4c17-aa88-81159454ace8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cd9b81ce48475aad2714e4b67a6496, user_id=22461a0d12b94583892357cb2be49866, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:32.670968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9dda89-cd92-4c17-aa88-81159454ace8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:32.691042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1071/2126] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:32 2022] PUT /identity/v3/projects/48cd9b81ce48475aad2714e4b67a6496/users/22461a0d12b94583892357cb2be49866/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:32.695648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce7205-3100-45fc-9853-25495e2b560a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.705892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce7205-3100-45fc-9853-25495e2b560a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:32.765178 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81676945-472b-432c-9369-0a57b99f5c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:32.765782 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81676945-472b-432c-9369-0a57b99f5c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.766067 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81676945-472b-432c-9369-0a57b99f5c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.766344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81676945-472b-432c-9369-0a57b99f5c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.767003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81676945-472b-432c-9369-0a57b99f5c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:32.768635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81676945-472b-432c-9369-0a57b99f5c13 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:32.779450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1056/2127] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:32.783497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dda89-cd92-4c17-aa88-81159454ace8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.794848 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dda89-cd92-4c17-aa88-81159454ace8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:32.799292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57efc87-8708-48f2-a245-475cab877b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:32.800349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57efc87-8708-48f2-a245-475cab877b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.800349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57efc87-8708-48f2-a245-475cab877b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.800661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57efc87-8708-48f2-a245-475cab877b79 admin admin] PATH_INFO: `/v3/projects/48cd9b81ce48475aad2714e4b67a6496/users/22461a0d12b94583892357cb2be49866/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.803029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57efc87-8708-48f2-a245-475cab877b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cd9b81ce48475aad2714e4b67a6496, user_id=22461a0d12b94583892357cb2be49866, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:32.805821 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57efc87-8708-48f2-a245-475cab877b79 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:32.818281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1072/2128] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:32 2022] PUT /identity/v3/projects/48cd9b81ce48475aad2714e4b67a6496/users/22461a0d12b94583892357cb2be49866/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:32.828593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9e68b-80a2-4020-aa47-f51a03c44320 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:32.828886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9e68b-80a2-4020-aa47-f51a03c44320 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:32.829167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9e68b-80a2-4020-aa47-f51a03c44320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:32.862965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a7c9e68b-80a2-4020-aa47-f51a03c44320 None None] MFA Rules not processed for user `22461a0d12b94583892357cb2be49866`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:32.864191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a7c9e68b-80a2-4020-aa47-f51a03c44320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:32.898605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:32.935823 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1057/2129] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:32.946737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57efc87-8708-48f2-a245-475cab877b79 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:32.948551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d57efc87-8708-48f2-a245-475cab877b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:33.006295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57efc87-8708-48f2-a245-475cab877b79 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:33.010155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2f84f-1621-420f-be31-aa558a482998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:33.010843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2f84f-1621-420f-be31-aa558a482998 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:33.011120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2f84f-1621-420f-be31-aa558a482998 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:33.011388 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2f84f-1621-420f-be31-aa558a482998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:33.068772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2f84f-1621-420f-be31-aa558a482998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:33.070525 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2f84f-1621-420f-be31-aa558a482998 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:33.079308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1073/2130] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:39.862949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9e68b-80a2-4020-aa47-f51a03c44320 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:39.927391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9e68b-80a2-4020-aa47-f51a03c44320 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:39.932030 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:39.932688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:39.932975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:39.933249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] PATH_INFO: `/v3/users/413d2d8e754d46feab6931702ba0496f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:39.934378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=413d2d8e754d46feab6931702ba0496f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:39.936308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:39.984860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '413d2d8e754d46feab6931702ba0496f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:39.985515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '413d2d8e754d46feab6931702ba0496f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:39.992649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '413d2d8e754d46feab6931702ba0496f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:39.999970 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '413d2d8e754d46feab6931702ba0496f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.000705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '413d2d8e754d46feab6931702ba0496f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.019664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1058/2131] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:39 2022] DELETE /identity/v3/users/413d2d8e754d46feab6931702ba0496f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:40.025395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bcf5d-2ecb-4345-941f-67d85a2e5773 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:40.025937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bcf5d-2ecb-4345-941f-67d85a2e5773 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:40.026103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bcf5d-2ecb-4345-941f-67d85a2e5773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:40.047214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-829bcf5d-2ecb-4345-941f-67d85a2e5773 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:40.048405 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-829bcf5d-2ecb-4345-941f-67d85a2e5773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:40.057344 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:40.112400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1074/2132] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:40.122696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:40.124285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:40.197724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:40.202742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d77c2-a50a-4529-a12e-88122f007b18 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:40.209619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05230296-3b92-48f6-b148-2b7243f12a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:40.210591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05230296-3b92-48f6-b148-2b7243f12a84 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:40.210905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05230296-3b92-48f6-b148-2b7243f12a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:40.211186 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05230296-3b92-48f6-b148-2b7243f12a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:40.221862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05230296-3b92-48f6-b148-2b7243f12a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:40.223303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05230296-3b92-48f6-b148-2b7243f12a84 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:40.234534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1059/2133] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:40.537453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bcf5d-2ecb-4345-941f-67d85a2e5773 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:40.539224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-829bcf5d-2ecb-4345-941f-67d85a2e5773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:40.555421 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bcf5d-2ecb-4345-941f-67d85a2e5773 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:40.569049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:40.569694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:40.569977 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:40.570255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] PATH_INFO: `/v3/projects/e269b10fde2d450798e4d36225a35351` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:40.571415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e269b10fde2d450798e4d36225a35351)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:40.573378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:40.625017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e269b10fde2d450798e4d36225a35351', 'request_id': 'req-a84d4c10-1ee5-490d-815a-dcb84f1803a4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.631955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e269b10fde2d450798e4d36225a35351', 'request_id': 'req-a84d4c10-1ee5-490d-815a-dcb84f1803a4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.637845 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e269b10fde2d450798e4d36225a35351', 'request_id': 'req-a84d4c10-1ee5-490d-815a-dcb84f1803a4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.652569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] The token cache is being invalidate because project e269b10fde2d450798e4d36225a35351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:40.652867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.653528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.655212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1075/2134] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:40 2022] DELETE /identity/v3/projects/e269b10fde2d450798e4d36225a35351 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:40.659228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05230296-3b92-48f6-b148-2b7243f12a84 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:40.669366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05230296-3b92-48f6-b148-2b7243f12a84 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:40.729329 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9547f-e25d-4b6d-881c-899b76462a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:40.729971 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9547f-e25d-4b6d-881c-899b76462a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:40.730238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9547f-e25d-4b6d-881c-899b76462a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:40.730515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9547f-e25d-4b6d-881c-899b76462a89 admin admin] PATH_INFO: `/v3/users/22461a0d12b94583892357cb2be49866` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:40.731661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9547f-e25d-4b6d-881c-899b76462a89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22461a0d12b94583892357cb2be49866)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:40.733620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9547f-e25d-4b6d-881c-899b76462a89 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:40.781448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c1d9547f-e25d-4b6d-881c-899b76462a89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22461a0d12b94583892357cb2be49866'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.782090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c1d9547f-e25d-4b6d-881c-899b76462a89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22461a0d12b94583892357cb2be49866'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.788931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c1d9547f-e25d-4b6d-881c-899b76462a89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22461a0d12b94583892357cb2be49866'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.796571 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c1d9547f-e25d-4b6d-881c-899b76462a89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22461a0d12b94583892357cb2be49866'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.797145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c1d9547f-e25d-4b6d-881c-899b76462a89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22461a0d12b94583892357cb2be49866'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:40.814425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1060/2135] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:40 2022] DELETE /identity/v3/users/22461a0d12b94583892357cb2be49866 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:41.084246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:41.085764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:41.098428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d4c10-1ee5-490d-815a-dcb84f1803a4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:41.158144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:41.158761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.159039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:41.159313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] PATH_INFO: `/v3/projects/48cd9b81ce48475aad2714e4b67a6496` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:41.160402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=48cd9b81ce48475aad2714e4b67a6496)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:41.162539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:41.208835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:41.217395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48cd9b81ce48475aad2714e4b67a6496', 'request_id': 'req-1985b871-5fac-4b29-9e06-870f0cdc4b3c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:41.225512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48cd9b81ce48475aad2714e4b67a6496', 'request_id': 'req-1985b871-5fac-4b29-9e06-870f0cdc4b3c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:41.233913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48cd9b81ce48475aad2714e4b67a6496', 'request_id': 'req-1985b871-5fac-4b29-9e06-870f0cdc4b3c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:41.258630 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] The token cache is being invalidate because project 48cd9b81ce48475aad2714e4b67a6496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:41.259047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:41.259637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1985b871-5fac-4b29-9e06-870f0cdc4b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:41.261459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1076/2136] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:41 2022] DELETE /identity/v3/projects/48cd9b81ce48475aad2714e4b67a6496 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:41.266173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d21914-c139-4dff-a196-0fac5414fe93 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.266708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d21914-c139-4dff-a196-0fac5414fe93 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:41.266794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d21914-c139-4dff-a196-0fac5414fe93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:41.288717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-91d21914-c139-4dff-a196-0fac5414fe93 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:41.290430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-91d21914-c139-4dff-a196-0fac5414fe93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:41.355614 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1061/2137] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:41.364706 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8910bf2-7311-41e0-8aa6-b5e523bb2d5e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.365011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8910bf2-7311-41e0-8aa6-b5e523bb2d5e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:41.365266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8910bf2-7311-41e0-8aa6-b5e523bb2d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:41.385886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e8910bf2-7311-41e0-8aa6-b5e523bb2d5e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:41.387132 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e8910bf2-7311-41e0-8aa6-b5e523bb2d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:41.396530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1077/2138] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:41.400864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d21914-c139-4dff-a196-0fac5414fe93 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:41.410747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d21914-c139-4dff-a196-0fac5414fe93 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:41.415637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c28c69-2806-48ec-896a-8c34b5b228ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:41.416470 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c28c69-2806-48ec-896a-8c34b5b228ba admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.417044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c28c69-2806-48ec-896a-8c34b5b228ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:41.417321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c28c69-2806-48ec-896a-8c34b5b228ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:41.417981 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c28c69-2806-48ec-896a-8c34b5b228ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:45:41.418289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c28c69-2806-48ec-896a-8c34b5b228ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:41.419958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c28c69-2806-48ec-896a-8c34b5b228ba admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:41.427941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:41.437082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1062/2139] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:45:41.438855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8910bf2-7311-41e0-8aa6-b5e523bb2d5e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:41.452654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8910bf2-7311-41e0-8aa6-b5e523bb2d5e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:41.457568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fde31-4716-4f68-a06a-d775a44ba0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:41.459017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fde31-4716-4f68-a06a-d775a44ba0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.459317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fde31-4716-4f68-a06a-d775a44ba0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:41.459592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fde31-4716-4f68-a06a-d775a44ba0a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:41.460245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fde31-4716-4f68-a06a-d775a44ba0a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:41.462201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fde31-4716-4f68-a06a-d775a44ba0a2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:41.495045 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1078/2140] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:41 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:41.496157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c28c69-2806-48ec-896a-8c34b5b228ba admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:41.506396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c28c69-2806-48ec-896a-8c34b5b228ba admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:41.563372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5657000-b964-4c32-aa6b-a91352ea3c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:41.563965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657000-b964-4c32-aa6b-a91352ea3c7a admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.564246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657000-b964-4c32-aa6b-a91352ea3c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:41.564546 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657000-b964-4c32-aa6b-a91352ea3c7a admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:41.565172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5657000-b964-4c32-aa6b-a91352ea3c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:41.567458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5657000-b964-4c32-aa6b-a91352ea3c7a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:41.588233 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1063/2141] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:41 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:41.592516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fde31-4716-4f68-a06a-d775a44ba0a2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:41.603163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fde31-4716-4f68-a06a-d775a44ba0a2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:41.607760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c92877-2df2-460e-a644-e1ce163a9832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:41.608291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c92877-2df2-460e-a644-e1ce163a9832 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.608607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c92877-2df2-460e-a644-e1ce163a9832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:41.608883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c92877-2df2-460e-a644-e1ce163a9832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:41.609523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c92877-2df2-460e-a644-e1ce163a9832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:41.611093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c92877-2df2-460e-a644-e1ce163a9832 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:41.620960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1079/2142] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:41.624858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5657000-b964-4c32-aa6b-a91352ea3c7a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:41.634937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5657000-b964-4c32-aa6b-a91352ea3c7a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:41.639464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea828b-3f32-458c-8283-c3fffc36c5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:41.640051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea828b-3f32-458c-8283-c3fffc36c5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.640331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea828b-3f32-458c-8283-c3fffc36c5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:41.640644 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea828b-3f32-458c-8283-c3fffc36c5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:41.641280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea828b-3f32-458c-8283-c3fffc36c5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:41.642917 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea828b-3f32-458c-8283-c3fffc36c5e0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:41.655597 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1064/2143] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:41.659790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c92877-2df2-460e-a644-e1ce163a9832 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:41.670449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c92877-2df2-460e-a644-e1ce163a9832 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:41.675249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e153038-421c-4a5a-8253-3aaf23a36b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:41.675900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e153038-421c-4a5a-8253-3aaf23a36b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.676178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e153038-421c-4a5a-8253-3aaf23a36b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:41.676448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e153038-421c-4a5a-8253-3aaf23a36b47 admin admin] PATH_INFO: `/v3/projects/f3cf8fd7d64a42f187885d46d6d43e6d/users/1c99bb45ee1e4baeaa7ac7578f8db8a4/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:41.692896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e153038-421c-4a5a-8253-3aaf23a36b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3cf8fd7d64a42f187885d46d6d43e6d, user_id=1c99bb45ee1e4baeaa7ac7578f8db8a4, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:41.695564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e153038-421c-4a5a-8253-3aaf23a36b47 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:41.710881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1080/2144] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:41 2022] PUT /identity/v3/projects/f3cf8fd7d64a42f187885d46d6d43e6d/users/1c99bb45ee1e4baeaa7ac7578f8db8a4/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:41.721176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dbaeb-a586-49d3-aac3-c66d66062620 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.721461 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dbaeb-a586-49d3-aac3-c66d66062620 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:41.721744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dbaeb-a586-49d3-aac3-c66d66062620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:41.749982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-360dbaeb-a586-49d3-aac3-c66d66062620 None None] MFA Rules not processed for user `1c99bb45ee1e4baeaa7ac7578f8db8a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:41.751257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-360dbaeb-a586-49d3-aac3-c66d66062620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:41.827863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1065/2145] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:41.832320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e153038-421c-4a5a-8253-3aaf23a36b47 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:41.893604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e153038-421c-4a5a-8253-3aaf23a36b47 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:41.898728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35d66a-3bf9-426f-886d-28c93e927ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:41.899316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35d66a-3bf9-426f-886d-28c93e927ebf admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.899594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35d66a-3bf9-426f-886d-28c93e927ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:41.899865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35d66a-3bf9-426f-886d-28c93e927ebf admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:41.900514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35d66a-3bf9-426f-886d-28c93e927ebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:41.902432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35d66a-3bf9-426f-886d-28c93e927ebf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:41.929237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1081/2146] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:41 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:41.933285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dbaeb-a586-49d3-aac3-c66d66062620 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:41.945192 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dbaeb-a586-49d3-aac3-c66d66062620 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:41.998965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee6f9a-c4d6-413c-93f8-5e903b9c25ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:41.999562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee6f9a-c4d6-413c-93f8-5e903b9c25ad admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:41.999841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee6f9a-c4d6-413c-93f8-5e903b9c25ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.000115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee6f9a-c4d6-413c-93f8-5e903b9c25ad admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.000772 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee6f9a-c4d6-413c-93f8-5e903b9c25ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:42.002687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee6f9a-c4d6-413c-93f8-5e903b9c25ad admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:42.014009 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:42.026251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1066/2147] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:41 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:42.030240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35d66a-3bf9-426f-886d-28c93e927ebf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:42.041594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35d66a-3bf9-426f-886d-28c93e927ebf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:42.046271 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b1283-6f29-4d63-be59-40b1e5394d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:42.046863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b1283-6f29-4d63-be59-40b1e5394d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.047147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b1283-6f29-4d63-be59-40b1e5394d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.047423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b1283-6f29-4d63-be59-40b1e5394d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.048061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b1283-6f29-4d63-be59-40b1e5394d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:42.049692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b1283-6f29-4d63-be59-40b1e5394d19 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:42.060987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1082/2148] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:42.064864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee6f9a-c4d6-413c-93f8-5e903b9c25ad admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:42.074956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee6f9a-c4d6-413c-93f8-5e903b9c25ad admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:42.079461 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51e4e5-1fae-4dd5-96d8-629726f62fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:42.080045 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e4e5-1fae-4dd5-96d8-629726f62fce admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.080327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e4e5-1fae-4dd5-96d8-629726f62fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.080638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e4e5-1fae-4dd5-96d8-629726f62fce admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.081270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51e4e5-1fae-4dd5-96d8-629726f62fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:42.083391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51e4e5-1fae-4dd5-96d8-629726f62fce admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:42.093072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1067/2149] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:42.097072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b1283-6f29-4d63-be59-40b1e5394d19 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:42.107529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b1283-6f29-4d63-be59-40b1e5394d19 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:42.112091 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512521f5-c0bd-4bb0-9117-6a4a905175d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:42.112759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512521f5-c0bd-4bb0-9117-6a4a905175d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.113040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512521f5-c0bd-4bb0-9117-6a4a905175d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.113399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512521f5-c0bd-4bb0-9117-6a4a905175d9 admin admin] PATH_INFO: `/v3/projects/04df3dad5a2945ff95623448925e2a9b/users/c9e99a24a0914e2c8b761a1b7c75c997/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.127641 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512521f5-c0bd-4bb0-9117-6a4a905175d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04df3dad5a2945ff95623448925e2a9b, user_id=c9e99a24a0914e2c8b761a1b7c75c997, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:42.130366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512521f5-c0bd-4bb0-9117-6a4a905175d9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:42.142971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1083/2150] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:42 2022] PUT /identity/v3/projects/04df3dad5a2945ff95623448925e2a9b/users/c9e99a24a0914e2c8b761a1b7c75c997/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:42.154299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecec7c-622a-4e28-8a8c-80ed4a4ff3a0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.154527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecec7c-622a-4e28-8a8c-80ed4a4ff3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.154829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecec7c-622a-4e28-8a8c-80ed4a4ff3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.182706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0fecec7c-622a-4e28-8a8c-80ed4a4ff3a0 None None] MFA Rules not processed for user `c9e99a24a0914e2c8b761a1b7c75c997`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:42.183926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0fecec7c-622a-4e28-8a8c-80ed4a4ff3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:42.250729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1068/2151] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:42.255314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512521f5-c0bd-4bb0-9117-6a4a905175d9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:42.272034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:42.331182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512521f5-c0bd-4bb0-9117-6a4a905175d9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:42.337833 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f66f41-04ea-474d-a102-49ef9fe056be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:42.338585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f66f41-04ea-474d-a102-49ef9fe056be admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.338871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f66f41-04ea-474d-a102-49ef9fe056be admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.339211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f66f41-04ea-474d-a102-49ef9fe056be admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.339950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f66f41-04ea-474d-a102-49ef9fe056be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:42.342025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f66f41-04ea-474d-a102-49ef9fe056be admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:42.376584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1084/2152] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:42 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:42.380563 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecec7c-622a-4e28-8a8c-80ed4a4ff3a0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:42.391112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecec7c-622a-4e28-8a8c-80ed4a4ff3a0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:42.447376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e0e2-c00b-4a31-876e-e13087fdd238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:42.447907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e0e2-c00b-4a31-876e-e13087fdd238 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.448191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e0e2-c00b-4a31-876e-e13087fdd238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.448468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e0e2-c00b-4a31-876e-e13087fdd238 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.449126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83e0e2-c00b-4a31-876e-e13087fdd238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:42.451088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83e0e2-c00b-4a31-876e-e13087fdd238 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:42.471868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1069/2153] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:42 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:42.475833 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f66f41-04ea-474d-a102-49ef9fe056be admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:42.486320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f66f41-04ea-474d-a102-49ef9fe056be admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:42.490883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c18c95-f99a-4b66-829d-a961f9f2dea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:42.491481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18c95-f99a-4b66-829d-a961f9f2dea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.491763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18c95-f99a-4b66-829d-a961f9f2dea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.492037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18c95-f99a-4b66-829d-a961f9f2dea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.492714 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c18c95-f99a-4b66-829d-a961f9f2dea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:42.494317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c18c95-f99a-4b66-829d-a961f9f2dea5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:42.508253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1085/2154] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:42.509416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e0e2-c00b-4a31-876e-e13087fdd238 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:42.519328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e0e2-c00b-4a31-876e-e13087fdd238 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:42.524924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e44ff-8d8b-4c70-bd88-b0aaec837034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:42.525588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e44ff-8d8b-4c70-bd88-b0aaec837034 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.525874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e44ff-8d8b-4c70-bd88-b0aaec837034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.526153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e44ff-8d8b-4c70-bd88-b0aaec837034 admin admin] PATH_INFO: `/v3/projects/bd915e77895a490ab00bc5c22fb311b9/users/871bcb3470a34b54bc7c036b528a3dfa/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.542118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e44ff-8d8b-4c70-bd88-b0aaec837034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd915e77895a490ab00bc5c22fb311b9, user_id=871bcb3470a34b54bc7c036b528a3dfa, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:42.544858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e44ff-8d8b-4c70-bd88-b0aaec837034 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:42.564168 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1070/2155] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:42 2022] PUT /identity/v3/projects/bd915e77895a490ab00bc5c22fb311b9/users/871bcb3470a34b54bc7c036b528a3dfa/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:42.568442 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c18c95-f99a-4b66-829d-a961f9f2dea5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:42.579716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c18c95-f99a-4b66-829d-a961f9f2dea5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:42.638447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4a4f5-66fb-445e-bf31-214743923a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:42.639046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4a4f5-66fb-445e-bf31-214743923a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.639320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4a4f5-66fb-445e-bf31-214743923a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.639721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4a4f5-66fb-445e-bf31-214743923a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.640363 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4a4f5-66fb-445e-bf31-214743923a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:42.641958 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4a4f5-66fb-445e-bf31-214743923a02 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:42.651671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1086/2156] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:42.656196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e44ff-8d8b-4c70-bd88-b0aaec837034 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:42.668089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e44ff-8d8b-4c70-bd88-b0aaec837034 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:42.672909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efab55-5835-4da1-9783-f9feb31d6763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:42.673566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efab55-5835-4da1-9783-f9feb31d6763 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.673903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efab55-5835-4da1-9783-f9feb31d6763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.674178 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efab55-5835-4da1-9783-f9feb31d6763 admin admin] PATH_INFO: `/v3/projects/bd915e77895a490ab00bc5c22fb311b9/users/871bcb3470a34b54bc7c036b528a3dfa/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.684699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efab55-5835-4da1-9783-f9feb31d6763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd915e77895a490ab00bc5c22fb311b9, user_id=871bcb3470a34b54bc7c036b528a3dfa, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:42.685062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efab55-5835-4da1-9783-f9feb31d6763 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:42.697702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1071/2157] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:42 2022] PUT /identity/v3/projects/bd915e77895a490ab00bc5c22fb311b9/users/871bcb3470a34b54bc7c036b528a3dfa/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:42.709718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bbae1-2a31-4669-93c7-5f72f0ebb9e5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.710004 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bbae1-2a31-4669-93c7-5f72f0ebb9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.710277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bbae1-2a31-4669-93c7-5f72f0ebb9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.753491 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9a4bbae1-2a31-4669-93c7-5f72f0ebb9e5 None None] MFA Rules not processed for user `871bcb3470a34b54bc7c036b528a3dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:42.755603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a4bbae1-2a31-4669-93c7-5f72f0ebb9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:42.841086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1087/2158] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:42.852885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efab55-5835-4da1-9783-f9feb31d6763 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:42.854760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-23efab55-5835-4da1-9783-f9feb31d6763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:42.890931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:42.922281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efab55-5835-4da1-9783-f9feb31d6763 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:42.926214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac68a72-2ad0-474c-8ea7-02cee989150c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:42.926939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac68a72-2ad0-474c-8ea7-02cee989150c service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:42.927319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac68a72-2ad0-474c-8ea7-02cee989150c service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:42.927776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac68a72-2ad0-474c-8ea7-02cee989150c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:42.995998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac68a72-2ad0-474c-8ea7-02cee989150c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:42.997522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac68a72-2ad0-474c-8ea7-02cee989150c service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:43.007953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1072/2159] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:44.996843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4bbae1-2a31-4669-93c7-5f72f0ebb9e5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:44.998245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a4bbae1-2a31-4669-93c7-5f72f0ebb9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:45.009166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:45.064249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4bbae1-2a31-4669-93c7-5f72f0ebb9e5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:45.069142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:45.069757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:45.070036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:45.070304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] PATH_INFO: `/v3/users/fe2b0a7ecdb0457184969a98c53f1599` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:45.071400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe2b0a7ecdb0457184969a98c53f1599)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:45.073352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:45.119622 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe2b0a7ecdb0457184969a98c53f1599'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:45.126661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe2b0a7ecdb0457184969a98c53f1599'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:45.127361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe2b0a7ecdb0457184969a98c53f1599'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:45.127972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe2b0a7ecdb0457184969a98c53f1599'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:45.135669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe2b0a7ecdb0457184969a98c53f1599'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:45.153580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1088/2160] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:44 2022] DELETE /identity/v3/users/fe2b0a7ecdb0457184969a98c53f1599 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:45.158041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea5d79-0b3c-4558-b713-0a6b0265e78a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:45.158506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea5d79-0b3c-4558-b713-0a6b0265e78a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:45.158718 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea5d79-0b3c-4558-b713-0a6b0265e78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:45.179015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f8ea5d79-0b3c-4558-b713-0a6b0265e78a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:45.180222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f8ea5d79-0b3c-4558-b713-0a6b0265e78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:45.241081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1073/2161] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:45.250799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:45.252813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:45.311035 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b711e2d-9663-47e1-9573-d1493175f208 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:45.314837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bd388-9ba6-463f-a9e8-0f224a64c2a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:45.315447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bd388-9ba6-463f-a9e8-0f224a64c2a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:45.315728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bd388-9ba6-463f-a9e8-0f224a64c2a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:45.316256 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bd388-9ba6-463f-a9e8-0f224a64c2a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:45.326162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6bd388-9ba6-463f-a9e8-0f224a64c2a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:45.328003 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6bd388-9ba6-463f-a9e8-0f224a64c2a5 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:45.339550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1089/2162] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:45.607966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea5d79-0b3c-4558-b713-0a6b0265e78a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:45.610060 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f8ea5d79-0b3c-4558-b713-0a6b0265e78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:45.649245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bd388-9ba6-463f-a9e8-0f224a64c2a5 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:45.650759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7c6bd388-9ba6-463f-a9e8-0f224a64c2a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:45.668207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bd388-9ba6-463f-a9e8-0f224a64c2a5 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:45.674626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea5d79-0b3c-4558-b713-0a6b0265e78a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:45.675882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:45.677113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:45.677375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:45.677652 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] PATH_INFO: `/v3/projects/3164dc1f6a1047f78dd20c649a10aed4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:45.678849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3164dc1f6a1047f78dd20c649a10aed4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:45.682941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:45.690155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf072e8-6805-4418-a4a7-20025fe96b62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:45.691151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf072e8-6805-4418-a4a7-20025fe96b62 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:45.691536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf072e8-6805-4418-a4a7-20025fe96b62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:45.691804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf072e8-6805-4418-a4a7-20025fe96b62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:45.694276 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-adf072e8-6805-4418-a4a7-20025fe96b62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:45.717879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:45.732078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:45.738550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3164dc1f6a1047f78dd20c649a10aed4', 'request_id': 'req-77453654-c0f9-4990-9fe5-48a7a417f602'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:45.745616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3164dc1f6a1047f78dd20c649a10aed4', 'request_id': 'req-77453654-c0f9-4990-9fe5-48a7a417f602'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:45.751619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3164dc1f6a1047f78dd20c649a10aed4', 'request_id': 'req-77453654-c0f9-4990-9fe5-48a7a417f602'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:45.758719 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf072e8-6805-4418-a4a7-20025fe96b62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:45.760278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf072e8-6805-4418-a4a7-20025fe96b62 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:45.764504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] The token cache is being invalidate because project 3164dc1f6a1047f78dd20c649a10aed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:45.764810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:45.765429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77453654-c0f9-4990-9fe5-48a7a417f602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:45.766933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1090/2163] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:45 2022] DELETE /identity/v3/projects/3164dc1f6a1047f78dd20c649a10aed4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:45.772190 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1074/2164] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:45.772387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ceeef-15bb-42d2-8add-0765531adba3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:45.772773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ceeef-15bb-42d2-8add-0765531adba3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:45.773052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ceeef-15bb-42d2-8add-0765531adba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:45.792788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8b6ceeef-15bb-42d2-8add-0765531adba3 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:45.793978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8b6ceeef-15bb-42d2-8add-0765531adba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:45.856051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1091/2165] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:45:45.866437 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0520b-3b24-481e-a726-1ff4a4e01a19 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:45.866725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0520b-3b24-481e-a726-1ff4a4e01a19 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:45.867005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0520b-3b24-481e-a726-1ff4a4e01a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:45.888229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-76c0520b-3b24-481e-a726-1ff4a4e01a19 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:45.889504 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-76c0520b-3b24-481e-a726-1ff4a4e01a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:45.898062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1075/2166] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:45.902346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ceeef-15bb-42d2-8add-0765531adba3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:45.912280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ceeef-15bb-42d2-8add-0765531adba3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:45.917757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56572f97-a17b-4f81-ac0e-7bf55a32ae55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:45.918891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56572f97-a17b-4f81-ac0e-7bf55a32ae55 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:45.919243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56572f97-a17b-4f81-ac0e-7bf55a32ae55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:45.919592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56572f97-a17b-4f81-ac0e-7bf55a32ae55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:45.920333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56572f97-a17b-4f81-ac0e-7bf55a32ae55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:45:45.920755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56572f97-a17b-4f81-ac0e-7bf55a32ae55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:45.922517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56572f97-a17b-4f81-ac0e-7bf55a32ae55 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:45.936102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1092/2167] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:45:45.940436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0520b-3b24-481e-a726-1ff4a4e01a19 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:45.951523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0520b-3b24-481e-a726-1ff4a4e01a19 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:45.958695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c5a40-e958-4ee7-b72c-f6b661a49c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:45.959312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c5a40-e958-4ee7-b72c-f6b661a49c61 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:45.959879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c5a40-e958-4ee7-b72c-f6b661a49c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:45.960159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c5a40-e958-4ee7-b72c-f6b661a49c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:45.960839 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c5a40-e958-4ee7-b72c-f6b661a49c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:45.963098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c5a40-e958-4ee7-b72c-f6b661a49c61 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:46.000611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1076/2168] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:45 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:46.003431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56572f97-a17b-4f81-ac0e-7bf55a32ae55 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:46.014026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56572f97-a17b-4f81-ac0e-7bf55a32ae55 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:46.074528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bbd0e-8da6-43dd-8c90-82b8aa7cb24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:46.075135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bbd0e-8da6-43dd-8c90-82b8aa7cb24f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:46.075419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bbd0e-8da6-43dd-8c90-82b8aa7cb24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:46.075691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bbd0e-8da6-43dd-8c90-82b8aa7cb24f admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:46.076362 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bbd0e-8da6-43dd-8c90-82b8aa7cb24f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:46.078336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bbd0e-8da6-43dd-8c90-82b8aa7cb24f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:46.103815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1093/2169] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:46 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:46.105060 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c5a40-e958-4ee7-b72c-f6b661a49c61 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:46.116870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c5a40-e958-4ee7-b72c-f6b661a49c61 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:46.121950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5cb8c-2b6b-4c2f-8be8-c3391be0f512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:46.122591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5cb8c-2b6b-4c2f-8be8-c3391be0f512 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:46.122875 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5cb8c-2b6b-4c2f-8be8-c3391be0f512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:46.123151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5cb8c-2b6b-4c2f-8be8-c3391be0f512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:46.123797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5cb8c-2b6b-4c2f-8be8-c3391be0f512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:46.125441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5cb8c-2b6b-4c2f-8be8-c3391be0f512 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:46.137518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1077/2170] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:46.141776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bbd0e-8da6-43dd-8c90-82b8aa7cb24f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:46.152202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bbd0e-8da6-43dd-8c90-82b8aa7cb24f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:46.157682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e7704-4ed7-4893-9bde-1f37682f5fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:46.158283 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e7704-4ed7-4893-9bde-1f37682f5fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:46.158571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e7704-4ed7-4893-9bde-1f37682f5fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:46.158844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e7704-4ed7-4893-9bde-1f37682f5fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:46.159497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e7704-4ed7-4893-9bde-1f37682f5fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:46.161200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e7704-4ed7-4893-9bde-1f37682f5fe9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:46.172141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1094/2171] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:46.176953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5cb8c-2b6b-4c2f-8be8-c3391be0f512 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:46.187452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5cb8c-2b6b-4c2f-8be8-c3391be0f512 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:46.193078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362879b7-dae0-4b72-a0ba-4ff3c4856228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:46.193925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362879b7-dae0-4b72-a0ba-4ff3c4856228 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:46.194099 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362879b7-dae0-4b72-a0ba-4ff3c4856228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:46.194379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362879b7-dae0-4b72-a0ba-4ff3c4856228 admin admin] PATH_INFO: `/v3/projects/e4e02b8409eb43e1bd1d68c79e985430/users/255f7abaa92444309e5bf97f9d91c840/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:46.209775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362879b7-dae0-4b72-a0ba-4ff3c4856228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e02b8409eb43e1bd1d68c79e985430, user_id=255f7abaa92444309e5bf97f9d91c840, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:46.212448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362879b7-dae0-4b72-a0ba-4ff3c4856228 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:46.228671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1078/2172] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:46 2022] PUT /identity/v3/projects/e4e02b8409eb43e1bd1d68c79e985430/users/255f7abaa92444309e5bf97f9d91c840/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:46.233815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fb78b-6666-4fbe-a12b-ff11563e114a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:46.234108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fb78b-6666-4fbe-a12b-ff11563e114a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:46.234389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fb78b-6666-4fbe-a12b-ff11563e114a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:46.257303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c13fb78b-6666-4fbe-a12b-ff11563e114a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:46.258614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c13fb78b-6666-4fbe-a12b-ff11563e114a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:46.272468 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:46.323312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362879b7-dae0-4b72-a0ba-4ff3c4856228 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:46.325085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-362879b7-dae0-4b72-a0ba-4ff3c4856228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:46.329061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1095/2173] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:46.339915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fb78b-6666-4fbe-a12b-ff11563e114a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:46.341114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c13fb78b-6666-4fbe-a12b-ff11563e114a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:46.386290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362879b7-dae0-4b72-a0ba-4ff3c4856228 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:46.389982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589336b9-277f-4b35-aff2-e42c6b709e63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:46.390695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589336b9-277f-4b35-aff2-e42c6b709e63 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:46.390879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589336b9-277f-4b35-aff2-e42c6b709e63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:46.391157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589336b9-277f-4b35-aff2-e42c6b709e63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:46.393375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-589336b9-277f-4b35-aff2-e42c6b709e63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:46.402904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fb78b-6666-4fbe-a12b-ff11563e114a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:46.406671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cb7e0-f971-4efc-848b-a1aeeffe3264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:46.407347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb7e0-f971-4efc-848b-a1aeeffe3264 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:46.407702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb7e0-f971-4efc-848b-a1aeeffe3264 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:46.408043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb7e0-f971-4efc-848b-a1aeeffe3264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:46.418676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cb7e0-f971-4efc-848b-a1aeeffe3264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:46.420122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cb7e0-f971-4efc-848b-a1aeeffe3264 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:46.430231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1096/2174] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:45:46.458477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589336b9-277f-4b35-aff2-e42c6b709e63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:46.459933 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589336b9-277f-4b35-aff2-e42c6b709e63 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:46.470407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1079/2175] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:45:50.647626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36efc7-bdc3-46c3-ad86-d360160be20b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:50.648192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36efc7-bdc3-46c3-ad86-d360160be20b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:50.648331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36efc7-bdc3-46c3-ad86-d360160be20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:50.666448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589336b9-277f-4b35-aff2-e42c6b709e63 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:50.667973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-589336b9-277f-4b35-aff2-e42c6b709e63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:50.684657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9e36efc7-bdc3-46c3-ad86-d360160be20b None None] MFA Rules not processed for user `255f7abaa92444309e5bf97f9d91c840`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:50.684657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9e36efc7-bdc3-46c3-ad86-d360160be20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:50.685787 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589336b9-277f-4b35-aff2-e42c6b709e63 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:50.694120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e2c86-e36b-4361-ba39-9d674bc1cf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:50.694868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e2c86-e36b-4361-ba39-9d674bc1cf4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:50.695219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e2c86-e36b-4361-ba39-9d674bc1cf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:50.695572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e2c86-e36b-4361-ba39-9d674bc1cf4d admin admin] PATH_INFO: `/v3/users/1c99bb45ee1e4baeaa7ac7578f8db8a4` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:50.696895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e2c86-e36b-4361-ba39-9d674bc1cf4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c99bb45ee1e4baeaa7ac7578f8db8a4)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:50.698944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e2c86-e36b-4361-ba39-9d674bc1cf4d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:50.718084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:50.759004 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1097/2176] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:50.760333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-607e2c86-e36b-4361-ba39-9d674bc1cf4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c99bb45ee1e4baeaa7ac7578f8db8a4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:50.761508 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-607e2c86-e36b-4361-ba39-9d674bc1cf4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c99bb45ee1e4baeaa7ac7578f8db8a4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:50.764207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36efc7-bdc3-46c3-ad86-d360160be20b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:50.779012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-607e2c86-e36b-4361-ba39-9d674bc1cf4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c99bb45ee1e4baeaa7ac7578f8db8a4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:50.786853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36efc7-bdc3-46c3-ad86-d360160be20b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:50.794551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-607e2c86-e36b-4361-ba39-9d674bc1cf4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c99bb45ee1e4baeaa7ac7578f8db8a4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:50.794844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d2b92-721c-4e69-aae5-8774a72263e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:50.795672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d2b92-721c-4e69-aae5-8774a72263e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:50.796150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d2b92-721c-4e69-aae5-8774a72263e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:50.796355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-607e2c86-e36b-4361-ba39-9d674bc1cf4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c99bb45ee1e4baeaa7ac7578f8db8a4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:50.797480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d2b92-721c-4e69-aae5-8774a72263e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:50.798335 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d2b92-721c-4e69-aae5-8774a72263e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:50.803192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d2b92-721c-4e69-aae5-8774a72263e8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:50.822453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1080/2177] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:50 2022] DELETE /identity/v3/users/1c99bb45ee1e4baeaa7ac7578f8db8a4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:50.828476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35589401-30f8-4cee-b6c5-990cbfb04eff None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:50.829014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35589401-30f8-4cee-b6c5-990cbfb04eff None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:50.829445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35589401-30f8-4cee-b6c5-990cbfb04eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:50.836614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1098/2178] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:50 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:50.841338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d2b92-721c-4e69-aae5-8774a72263e8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:50.843224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0f5d2b92-721c-4e69-aae5-8774a72263e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:50.859636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-35589401-30f8-4cee-b6c5-990cbfb04eff None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:50.861159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-35589401-30f8-4cee-b6c5-990cbfb04eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:50.892020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:50.932566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1081/2179] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:45:50.934488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d2b92-721c-4e69-aae5-8774a72263e8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:50.940709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f580441-601c-451c-915a-3ba4ef229b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:50.941725 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f580441-601c-451c-915a-3ba4ef229b1f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:50.942198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f580441-601c-451c-915a-3ba4ef229b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:50.942823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f580441-601c-451c-915a-3ba4ef229b1f admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:50.943725 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f580441-601c-451c-915a-3ba4ef229b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:50.945562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35589401-30f8-4cee-b6c5-990cbfb04eff None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:50.945982 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f580441-601c-451c-915a-3ba4ef229b1f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:50.948586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-35589401-30f8-4cee-b6c5-990cbfb04eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:50.972936 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1099/2180] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:50 2022] POST /identity/v3/users => generated 298 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:50.977457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f580441-601c-451c-915a-3ba4ef229b1f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:51.003438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f580441-601c-451c-915a-3ba4ef229b1f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:51.020216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b843f14-28c9-4c70-a647-fb420feff8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:51.035057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b843f14-28c9-4c70-a647-fb420feff8ee admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:51.036632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b843f14-28c9-4c70-a647-fb420feff8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:51.037073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b843f14-28c9-4c70-a647-fb420feff8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:51.038165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b843f14-28c9-4c70-a647-fb420feff8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:51.044873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b843f14-28c9-4c70-a647-fb420feff8ee admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:51.047147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35589401-30f8-4cee-b6c5-990cbfb04eff None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:51.058346 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c3171-30f2-4cf0-a195-bff897ed3a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:51.059013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c3171-30f2-4cf0-a195-bff897ed3a88 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:51.059300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c3171-30f2-4cf0-a195-bff897ed3a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:51.059575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c3171-30f2-4cf0-a195-bff897ed3a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:51.061907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1100/2181] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:45:51.066516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b843f14-28c9-4c70-a647-fb420feff8ee admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:51.070041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c3171-30f2-4cf0-a195-bff897ed3a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:51.071504 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c3171-30f2-4cf0-a195-bff897ed3a88 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:51.077653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b843f14-28c9-4c70-a647-fb420feff8ee admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:51.087179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1082/2182] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:51.088094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487c470-2818-4b8d-a6ea-34bcc1e9fff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:51.088810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487c470-2818-4b8d-a6ea-34bcc1e9fff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:51.090541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487c470-2818-4b8d-a6ea-34bcc1e9fff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:51.090882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487c470-2818-4b8d-a6ea-34bcc1e9fff5 admin admin] PATH_INFO: `/v3/projects/9bd9bcd609ef4b968aedb45b7052a897/users/eb61e69d574c494da009a6925893213a/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:51.111589 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487c470-2818-4b8d-a6ea-34bcc1e9fff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd9bcd609ef4b968aedb45b7052a897, user_id=eb61e69d574c494da009a6925893213a, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:51.115413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487c470-2818-4b8d-a6ea-34bcc1e9fff5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:51.154996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1101/2183] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:51 2022] PUT /identity/v3/projects/9bd9bcd609ef4b968aedb45b7052a897/users/eb61e69d574c494da009a6925893213a/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:51.159662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c3171-30f2-4cf0-a195-bff897ed3a88 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:51.183398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c3171-30f2-4cf0-a195-bff897ed3a88 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:51.270319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:51.324258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70033a4e-f009-4645-b1fd-c88e2a6ed04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:51.324922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70033a4e-f009-4645-b1fd-c88e2a6ed04f admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:51.325350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70033a4e-f009-4645-b1fd-c88e2a6ed04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:51.325608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70033a4e-f009-4645-b1fd-c88e2a6ed04f admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:51.326939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70033a4e-f009-4645-b1fd-c88e2a6ed04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:51.328475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70033a4e-f009-4645-b1fd-c88e2a6ed04f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:51.340962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1083/2184] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:51.347266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487c470-2818-4b8d-a6ea-34bcc1e9fff5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:51.362201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487c470-2818-4b8d-a6ea-34bcc1e9fff5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:51.375949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23e6a0-6b70-434b-b097-11250f7a3650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:51.376853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23e6a0-6b70-434b-b097-11250f7a3650 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:51.377133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23e6a0-6b70-434b-b097-11250f7a3650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:51.377406 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23e6a0-6b70-434b-b097-11250f7a3650 admin admin] PATH_INFO: `/v3/projects/9bd9bcd609ef4b968aedb45b7052a897/users/eb61e69d574c494da009a6925893213a/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:51.383171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23e6a0-6b70-434b-b097-11250f7a3650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd9bcd609ef4b968aedb45b7052a897, user_id=eb61e69d574c494da009a6925893213a, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:51.389688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23e6a0-6b70-434b-b097-11250f7a3650 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:51.422176 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1102/2185] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:51 2022] PUT /identity/v3/projects/9bd9bcd609ef4b968aedb45b7052a897/users/eb61e69d574c494da009a6925893213a/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:51.539941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70033a4e-f009-4645-b1fd-c88e2a6ed04f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:51.541760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-70033a4e-f009-4645-b1fd-c88e2a6ed04f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:51.558441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70033a4e-f009-4645-b1fd-c88e2a6ed04f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:51.651829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:51.652477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:51.652785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:51.653060 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] PATH_INFO: `/v3/projects/f3cf8fd7d64a42f187885d46d6d43e6d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:51.654207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3cf8fd7d64a42f187885d46d6d43e6d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:51.656390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:51.761665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3cf8fd7d64a42f187885d46d6d43e6d', 'request_id': 'req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:51.773913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3cf8fd7d64a42f187885d46d6d43e6d', 'request_id': 'req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:51.781660 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3cf8fd7d64a42f187885d46d6d43e6d', 'request_id': 'req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:51.800310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] The token cache is being invalidate because project f3cf8fd7d64a42f187885d46d6d43e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:51.800826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:51.802158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:51.803971 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1084/2186] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:51 2022] DELETE /identity/v3/projects/f3cf8fd7d64a42f187885d46d6d43e6d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:51.808484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23e6a0-6b70-434b-b097-11250f7a3650 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:51.811582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bc23e6a0-6b70-434b-b097-11250f7a3650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:51.830355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23e6a0-6b70-434b-b097-11250f7a3650 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:51.908147 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:51.908991 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:51.912570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:51.912570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] PATH_INFO: `/v3/users/c9e99a24a0914e2c8b761a1b7c75c997` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:51.917514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9e99a24a0914e2c8b761a1b7c75c997)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:51.919618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:51.986379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9e99a24a0914e2c8b761a1b7c75c997'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:51.994585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9e99a24a0914e2c8b761a1b7c75c997'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:51.995305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9e99a24a0914e2c8b761a1b7c75c997'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:51.995912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9e99a24a0914e2c8b761a1b7c75c997'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.007921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9e99a24a0914e2c8b761a1b7c75c997'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.019332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:52.032557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1103/2187] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:51 2022] DELETE /identity/v3/users/c9e99a24a0914e2c8b761a1b7c75c997 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:52.371870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:52.372579 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:52.374570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:52.375127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:52.404421 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b9d7c-930b-4ea7-95f2-b50b05baa096 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:52.477400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d8bb8-3f3d-4d4a-9dc1-8b0ef2bc7f15 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:52.485072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:52.485996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:52.486283 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:52.486933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] PATH_INFO: `/v3/projects/04df3dad5a2945ff95623448925e2a9b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:52.488600 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04df3dad5a2945ff95623448925e2a9b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:52.491970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:52.515143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19266445-2e2f-42cf-a937-8302f44fc4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:52.515892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19266445-2e2f-42cf-a937-8302f44fc4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:52.516249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19266445-2e2f-42cf-a937-8302f44fc4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:52.516633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19266445-2e2f-42cf-a937-8302f44fc4fc admin admin] PATH_INFO: `/v3/users/00f153678d3547ccbe0bbbe3df93ed9d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:52.518032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19266445-2e2f-42cf-a937-8302f44fc4fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f153678d3547ccbe0bbbe3df93ed9d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:52.520163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19266445-2e2f-42cf-a937-8302f44fc4fc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:52.532205 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:52.587989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04df3dad5a2945ff95623448925e2a9b', 'request_id': 'req-debe030a-330a-4ade-b439-8a5a1573ff86'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.596166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04df3dad5a2945ff95623448925e2a9b', 'request_id': 'req-debe030a-330a-4ade-b439-8a5a1573ff86'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.603344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-19266445-2e2f-42cf-a937-8302f44fc4fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f153678d3547ccbe0bbbe3df93ed9d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.604165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04df3dad5a2945ff95623448925e2a9b', 'request_id': 'req-debe030a-330a-4ade-b439-8a5a1573ff86'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.604639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-19266445-2e2f-42cf-a937-8302f44fc4fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f153678d3547ccbe0bbbe3df93ed9d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.615485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-19266445-2e2f-42cf-a937-8302f44fc4fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f153678d3547ccbe0bbbe3df93ed9d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.625476 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] The token cache is being invalidate because project 04df3dad5a2945ff95623448925e2a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:52.625773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.626726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-19266445-2e2f-42cf-a937-8302f44fc4fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f153678d3547ccbe0bbbe3df93ed9d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.627133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.627516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-19266445-2e2f-42cf-a937-8302f44fc4fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f153678d3547ccbe0bbbe3df93ed9d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.629560 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1104/2188] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:52 2022] DELETE /identity/v3/projects/04df3dad5a2945ff95623448925e2a9b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:52.634408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:52.636041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:52.668317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1085/2189] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:52 2022] DELETE /identity/v3/users/00f153678d3547ccbe0bbbe3df93ed9d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:52.670040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041211c2-bd6c-4195-a775-8fa698cd1ce5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:52.670337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041211c2-bd6c-4195-a775-8fa698cd1ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:52.670613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041211c2-bd6c-4195-a775-8fa698cd1ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:52.701625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-041211c2-bd6c-4195-a775-8fa698cd1ce5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:52.703212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-041211c2-bd6c-4195-a775-8fa698cd1ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:52.738127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe030a-330a-4ade-b439-8a5a1573ff86 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:52.768171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:52.800991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1086/2190] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:52.813253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041211c2-bd6c-4195-a775-8fa698cd1ce5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:52.814771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-041211c2-bd6c-4195-a775-8fa698cd1ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:52.825650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e77c0-5703-4b0a-8497-bdaca2940dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:52.826309 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e77c0-5703-4b0a-8497-bdaca2940dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:52.826596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e77c0-5703-4b0a-8497-bdaca2940dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:52.826871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e77c0-5703-4b0a-8497-bdaca2940dee admin admin] PATH_INFO: `/v3/users/871bcb3470a34b54bc7c036b528a3dfa` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:52.828145 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e77c0-5703-4b0a-8497-bdaca2940dee admin admin] RBAC: Authorizing `identity:delete_user(user_id=871bcb3470a34b54bc7c036b528a3dfa)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:52.830271 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e77c0-5703-4b0a-8497-bdaca2940dee admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:52.914904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-569e77c0-5703-4b0a-8497-bdaca2940dee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '871bcb3470a34b54bc7c036b528a3dfa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.915490 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041211c2-bd6c-4195-a775-8fa698cd1ce5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:52.922170 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7bb64-b897-4523-b713-cbf1b1c1ab53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:52.923570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7bb64-b897-4523-b713-cbf1b1c1ab53 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:52.923914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7bb64-b897-4523-b713-cbf1b1c1ab53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:52.924201 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7bb64-b897-4523-b713-cbf1b1c1ab53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:52.924462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-569e77c0-5703-4b0a-8497-bdaca2940dee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '871bcb3470a34b54bc7c036b528a3dfa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.925352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-569e77c0-5703-4b0a-8497-bdaca2940dee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '871bcb3470a34b54bc7c036b528a3dfa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.927324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-569e77c0-5703-4b0a-8497-bdaca2940dee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '871bcb3470a34b54bc7c036b528a3dfa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.936680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-569e77c0-5703-4b0a-8497-bdaca2940dee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '871bcb3470a34b54bc7c036b528a3dfa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:52.937147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7bb64-b897-4523-b713-cbf1b1c1ab53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:52.939041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7bb64-b897-4523-b713-cbf1b1c1ab53 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:52.951560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1087/2191] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:45:52.965931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1105/2192] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:52 2022] DELETE /identity/v3/users/871bcb3470a34b54bc7c036b528a3dfa => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:53.449847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7bb64-b897-4523-b713-cbf1b1c1ab53 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:53.462428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7bb64-b897-4523-b713-cbf1b1c1ab53 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:53.515575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:53.606947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:53.607947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:53.608373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:53.608832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] PATH_INFO: `/v3/projects/bd915e77895a490ab00bc5c22fb311b9` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:53.613158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd915e77895a490ab00bc5c22fb311b9)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:53.615853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:53.715465 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd915e77895a490ab00bc5c22fb311b9', 'request_id': 'req-a8893236-df84-48f9-91b3-c613e503632f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:53.724709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd915e77895a490ab00bc5c22fb311b9', 'request_id': 'req-a8893236-df84-48f9-91b3-c613e503632f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:53.737341 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd915e77895a490ab00bc5c22fb311b9', 'request_id': 'req-a8893236-df84-48f9-91b3-c613e503632f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:53.772030 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] The token cache is being invalidate because project bd915e77895a490ab00bc5c22fb311b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:53.773206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:53.773498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:53.781129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1088/2193] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:53 2022] DELETE /identity/v3/projects/bd915e77895a490ab00bc5c22fb311b9 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:53.783085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d9ba-db02-4f4c-860c-a47560f24fc7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:53.783379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d9ba-db02-4f4c-860c-a47560f24fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:53.783658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d9ba-db02-4f4c-860c-a47560f24fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:53.786517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:53.788066 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:53.832436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0ce0d9ba-db02-4f4c-860c-a47560f24fc7 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:53.836590 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0ce0d9ba-db02-4f4c-860c-a47560f24fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:53.881227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8893236-df84-48f9-91b3-c613e503632f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:53.886789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:53.887431 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:53.887714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:53.887991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] PATH_INFO: `/v3/projects/444aa9c6ee34483587001e470f88cc17` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:53.891116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444aa9c6ee34483587001e470f88cc17)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:53.897274 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:53.930140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1106/2194] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:53.938755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351da65-0a66-49fa-ba0e-9ce9fa53f51e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:53.939038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351da65-0a66-49fa-ba0e-9ce9fa53f51e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:53.939451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351da65-0a66-49fa-ba0e-9ce9fa53f51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:53.968015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4351da65-0a66-49fa-ba0e-9ce9fa53f51e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:53.969240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4351da65-0a66-49fa-ba0e-9ce9fa53f51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:53.969582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444aa9c6ee34483587001e470f88cc17', 'request_id': 'req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:53.976481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444aa9c6ee34483587001e470f88cc17', 'request_id': 'req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:53.978695 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:53.985075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444aa9c6ee34483587001e470f88cc17', 'request_id': 'req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:54.010264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] The token cache is being invalidate because project 444aa9c6ee34483587001e470f88cc17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:54.011445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:54.012656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7cc1e0c1-3316-46cc-8e29-daeafd2d4128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:54.018593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1089/2195] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:53 2022] DELETE /identity/v3/projects/444aa9c6ee34483587001e470f88cc17 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:45:54.021547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20982ecb-bb3e-4d0e-a517-8f21b86d0b92 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.022099 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20982ecb-bb3e-4d0e-a517-8f21b86d0b92 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.022584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20982ecb-bb3e-4d0e-a517-8f21b86d0b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.038384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:54.056170 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-20982ecb-bb3e-4d0e-a517-8f21b86d0b92 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:54.057916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-20982ecb-bb3e-4d0e-a517-8f21b86d0b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:54.066397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1107/2196] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:54.067748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351da65-0a66-49fa-ba0e-9ce9fa53f51e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.068441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1090/2197] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:54.076239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08490323-bdf0-4b64-997e-8e4d178c3ebc None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.076570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08490323-bdf0-4b64-997e-8e4d178c3ebc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.077239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08490323-bdf0-4b64-997e-8e4d178c3ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.081530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351da65-0a66-49fa-ba0e-9ce9fa53f51e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.088710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb70a2-4e6f-4f93-bada-a596cafd8cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.089587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb70a2-4e6f-4f93-bada-a596cafd8cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.090030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb70a2-4e6f-4f93-bada-a596cafd8cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.090392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb70a2-4e6f-4f93-bada-a596cafd8cbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.091142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb70a2-4e6f-4f93-bada-a596cafd8cbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:45:54.091540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb70a2-4e6f-4f93-bada-a596cafd8cbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.094110 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb70a2-4e6f-4f93-bada-a596cafd8cbd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.099397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-08490323-bdf0-4b64-997e-8e4d178c3ebc None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:54.100729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-08490323-bdf0-4b64-997e-8e4d178c3ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:54.114613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1091/2198] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:54.115647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08490323-bdf0-4b64-997e-8e4d178c3ebc None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.117765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1108/2199] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:45:54.122242 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb70a2-4e6f-4f93-bada-a596cafd8cbd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.130550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08490323-bdf0-4b64-997e-8e4d178c3ebc None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.137209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb96e29-5f57-44e3-9369-ca171607b76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.137594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb70a2-4e6f-4f93-bada-a596cafd8cbd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.137826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb96e29-5f57-44e3-9369-ca171607b76d admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.138110 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb96e29-5f57-44e3-9369-ca171607b76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.138564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb96e29-5f57-44e3-9369-ca171607b76d admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.139227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb96e29-5f57-44e3-9369-ca171607b76d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:45:54.139533 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb96e29-5f57-44e3-9369-ca171607b76d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.141605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb96e29-5f57-44e3-9369-ca171607b76d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.144099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35103de6-4b85-411a-b28d-8b009c21567a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.145552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35103de6-4b85-411a-b28d-8b009c21567a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.146818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35103de6-4b85-411a-b28d-8b009c21567a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.147134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35103de6-4b85-411a-b28d-8b009c21567a admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.147726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35103de6-4b85-411a-b28d-8b009c21567a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.149658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35103de6-4b85-411a-b28d-8b009c21567a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.163469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1092/2200] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:45:54.165921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb96e29-5f57-44e3-9369-ca171607b76d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.179350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb96e29-5f57-44e3-9369-ca171607b76d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.184626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c7c75-efe1-4109-8846-41d0a7608ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.185333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c7c75-efe1-4109-8846-41d0a7608ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.185595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c7c75-efe1-4109-8846-41d0a7608ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.185998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c7c75-efe1-4109-8846-41d0a7608ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.187756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c7c75-efe1-4109-8846-41d0a7608ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.188077 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1109/2201] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:54 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:45:54.189219 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35103de6-4b85-411a-b28d-8b009c21567a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.189811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c7c75-efe1-4109-8846-41d0a7608ff8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.248932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1093/2202] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:54 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:54.249923 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c7c75-efe1-4109-8846-41d0a7608ff8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.319667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35103de6-4b85-411a-b28d-8b009c21567a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.345722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c7c75-efe1-4109-8846-41d0a7608ff8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.360880 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a58177-a81b-43b3-be3e-3cb7eace3dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.361754 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a58177-a81b-43b3-be3e-3cb7eace3dba admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.361870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a58177-a81b-43b3-be3e-3cb7eace3dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.362060 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a58177-a81b-43b3-be3e-3cb7eace3dba admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.362993 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a58177-a81b-43b3-be3e-3cb7eace3dba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.366087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a58177-a81b-43b3-be3e-3cb7eace3dba admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.410743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1094/2203] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:54 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:54.418609 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a58177-a81b-43b3-be3e-3cb7eace3dba admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.454796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a58177-a81b-43b3-be3e-3cb7eace3dba admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.468053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805204a3-1c0f-418e-8ba8-5ff40ec41ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.468549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805204a3-1c0f-418e-8ba8-5ff40ec41ebc admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.468897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805204a3-1c0f-418e-8ba8-5ff40ec41ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.469553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805204a3-1c0f-418e-8ba8-5ff40ec41ebc admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.469804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805204a3-1c0f-418e-8ba8-5ff40ec41ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.471757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805204a3-1c0f-418e-8ba8-5ff40ec41ebc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.473795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a24059-4a35-4f97-baa2-0ea2701dc169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.474658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a24059-4a35-4f97-baa2-0ea2701dc169 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.474932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a24059-4a35-4f97-baa2-0ea2701dc169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.475200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a24059-4a35-4f97-baa2-0ea2701dc169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.475856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a24059-4a35-4f97-baa2-0ea2701dc169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.477491 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a24059-4a35-4f97-baa2-0ea2701dc169 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.497136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1095/2204] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:54.498952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a24059-4a35-4f97-baa2-0ea2701dc169 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.508178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1110/2205] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:54 2022] POST /identity/v3/users => generated 292 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:54.511436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a24059-4a35-4f97-baa2-0ea2701dc169 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.513716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805204a3-1c0f-418e-8ba8-5ff40ec41ebc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.517796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ce599-2132-400f-8af1-38a2dbc52037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.518537 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ce599-2132-400f-8af1-38a2dbc52037 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.519003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ce599-2132-400f-8af1-38a2dbc52037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.519305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ce599-2132-400f-8af1-38a2dbc52037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.520300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ce599-2132-400f-8af1-38a2dbc52037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.522094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ce599-2132-400f-8af1-38a2dbc52037 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.532952 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805204a3-1c0f-418e-8ba8-5ff40ec41ebc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.534423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1096/2206] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:54.539177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ce599-2132-400f-8af1-38a2dbc52037 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.556674 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b90b36-07f0-4fc7-8948-46126367f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.557066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b90b36-07f0-4fc7-8948-46126367f33f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.557349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b90b36-07f0-4fc7-8948-46126367f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.557620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b90b36-07f0-4fc7-8948-46126367f33f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.558356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b90b36-07f0-4fc7-8948-46126367f33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.559893 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b90b36-07f0-4fc7-8948-46126367f33f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.570271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ce599-2132-400f-8af1-38a2dbc52037 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.572553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1111/2207] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:54.577541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b90b36-07f0-4fc7-8948-46126367f33f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.587840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70d31e-7ce0-426d-bb50-e146a9eabfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.589265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b90b36-07f0-4fc7-8948-46126367f33f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.589434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70d31e-7ce0-426d-bb50-e146a9eabfba admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.589817 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70d31e-7ce0-426d-bb50-e146a9eabfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.590183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70d31e-7ce0-426d-bb50-e146a9eabfba admin admin] PATH_INFO: `/v3/projects/f8f83820c15b492c9013ff8345668b63/users/007760fc26c64cc9a9a3e6ea4325af31/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.596545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bbf00-9df5-4f23-a10c-fce02815b3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.597212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bbf00-9df5-4f23-a10c-fce02815b3df admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.597579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bbf00-9df5-4f23-a10c-fce02815b3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.599364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bbf00-9df5-4f23-a10c-fce02815b3df admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.599364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bbf00-9df5-4f23-a10c-fce02815b3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.601686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bbf00-9df5-4f23-a10c-fce02815b3df admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.618239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1112/2208] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:45:54.623586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bbf00-9df5-4f23-a10c-fce02815b3df admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.633874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70d31e-7ce0-426d-bb50-e146a9eabfba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8f83820c15b492c9013ff8345668b63, user_id=007760fc26c64cc9a9a3e6ea4325af31, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.636968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70d31e-7ce0-426d-bb50-e146a9eabfba admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.647243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bbf00-9df5-4f23-a10c-fce02815b3df admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:54.652964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e25f66-b29c-465f-ae2e-ab93a8790c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:54.653646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25f66-b29c-465f-ae2e-ab93a8790c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.653937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25f66-b29c-465f-ae2e-ab93a8790c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.654202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25f66-b29c-465f-ae2e-ab93a8790c87 admin admin] PATH_INFO: `/v3/projects/0d743e4a520b446ba88886cdd6c376fc/users/d4e1190f9c6e47e5924b1de7355a90a9/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.664597 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1097/2209] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:54 2022] PUT /identity/v3/projects/f8f83820c15b492c9013ff8345668b63/users/007760fc26c64cc9a9a3e6ea4325af31/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:54.668029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1254060-5172-4f8a-b969-e72dfacbe467 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.668435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1254060-5172-4f8a-b969-e72dfacbe467 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.668793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1254060-5172-4f8a-b969-e72dfacbe467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.672399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e25f66-b29c-465f-ae2e-ab93a8790c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d743e4a520b446ba88886cdd6c376fc, user_id=d4e1190f9c6e47e5924b1de7355a90a9, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:54.692696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e25f66-b29c-465f-ae2e-ab93a8790c87 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:54.719291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c1254060-5172-4f8a-b969-e72dfacbe467 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:54.720996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1113/2210] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:45:54 2022] PUT /identity/v3/projects/0d743e4a520b446ba88886cdd6c376fc/users/d4e1190f9c6e47e5924b1de7355a90a9/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:54.721919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c1254060-5172-4f8a-b969-e72dfacbe467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:54.740675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98602c-a3e4-410d-8641-2f09e66d609c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:54.742036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98602c-a3e4-410d-8641-2f09e66d609c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:54.742346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98602c-a3e4-410d-8641-2f09e66d609c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:54.764095 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:54.819006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:54.821727 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-aa98602c-a3e4-410d-8641-2f09e66d609c None None] MFA Rules not processed for user `d4e1190f9c6e47e5924b1de7355a90a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:54.823009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aa98602c-a3e4-410d-8641-2f09e66d609c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:54.878204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1098/2211] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:54.884362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1254060-5172-4f8a-b969-e72dfacbe467 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:54.888603 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c1254060-5172-4f8a-b969-e72dfacbe467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:54.987517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1114/2212] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:55.000085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98602c-a3e4-410d-8641-2f09e66d609c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:55.001488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aa98602c-a3e4-410d-8641-2f09e66d609c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:55.019307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1254060-5172-4f8a-b969-e72dfacbe467 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:55.029535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963c997-beab-4a45-9d63-efb9f196ac24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:55.030294 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963c997-beab-4a45-9d63-efb9f196ac24 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:55.030582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963c997-beab-4a45-9d63-efb9f196ac24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:55.030862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963c997-beab-4a45-9d63-efb9f196ac24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:55.064778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963c997-beab-4a45-9d63-efb9f196ac24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:55.066849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963c997-beab-4a45-9d63-efb9f196ac24 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:55.104784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1099/2213] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:55.169793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98602c-a3e4-410d-8641-2f09e66d609c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:55.187758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60589ed-af39-46ce-b003-97204158d8e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:55.188398 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60589ed-af39-46ce-b003-97204158d8e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:55.191302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60589ed-af39-46ce-b003-97204158d8e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:55.191546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60589ed-af39-46ce-b003-97204158d8e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:55.218881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60589ed-af39-46ce-b003-97204158d8e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:55.226448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60589ed-af39-46ce-b003-97204158d8e9 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:55.255298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1115/2214] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:56.787209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20fb18-601a-4162-95b0-4374f322f584 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:56.788938 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20fb18-601a-4162-95b0-4374f322f584 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:56.789715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20fb18-601a-4162-95b0-4374f322f584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:56.846086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ca20fb18-601a-4162-95b0-4374f322f584 None None] MFA Rules not processed for user `007760fc26c64cc9a9a3e6ea4325af31`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:56.848781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ca20fb18-601a-4162-95b0-4374f322f584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:57.025242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1100/2215] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:57.035587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60589ed-af39-46ce-b003-97204158d8e9 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:57.044423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60589ed-af39-46ce-b003-97204158d8e9 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:57.058177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92eeda-8595-4fed-9d88-8c19c2e2be8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:57.058797 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92eeda-8595-4fed-9d88-8c19c2e2be8b service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:57.059074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92eeda-8595-4fed-9d88-8c19c2e2be8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:57.059345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92eeda-8595-4fed-9d88-8c19c2e2be8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:57.079038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92eeda-8595-4fed-9d88-8c19c2e2be8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:57.079917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92eeda-8595-4fed-9d88-8c19c2e2be8b service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:57.104713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1116/2216] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:57.837909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663deb0-64be-4676-a1c8-be9e227523f7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:57.838493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663deb0-64be-4676-a1c8-be9e227523f7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:57.838493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663deb0-64be-4676-a1c8-be9e227523f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:57.870244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f663deb0-64be-4676-a1c8-be9e227523f7 None None] MFA Rules not processed for user `eb61e69d574c494da009a6925893213a`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:57.871488 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f663deb0-64be-4676-a1c8-be9e227523f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:57.941435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:57.945749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1101/2217] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:57.958404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92eeda-8595-4fed-9d88-8c19c2e2be8b service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:57.959955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1c92eeda-8595-4fed-9d88-8c19c2e2be8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:58.016942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92eeda-8595-4fed-9d88-8c19c2e2be8b service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:58.021689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fe401-ac05-498a-9b35-ee45a00da12a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ec048788904bc58245e878a8644091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:58.022292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fe401-ac05-498a-9b35-ee45a00da12a service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:58.022570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fe401-ac05-498a-9b35-ee45a00da12a service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:58.022838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fe401-ac05-498a-9b35-ee45a00da12a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:58.024696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-492fe401-ac05-498a-9b35-ee45a00da12a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:58.067422 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:58.089510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fe401-ac05-498a-9b35-ee45a00da12a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:58.090941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fe401-ac05-498a-9b35-ee45a00da12a service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:58.101931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1117/2218] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:58.308345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663deb0-64be-4676-a1c8-be9e227523f7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:58.316320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d5a31-babf-4582-8a68-0ca6a9dbf0b8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:58.317700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d5a31-babf-4582-8a68-0ca6a9dbf0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:58.318118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d5a31-babf-4582-8a68-0ca6a9dbf0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:58.326421 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663deb0-64be-4676-a1c8-be9e227523f7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:58.335152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc59116-ca38-4d27-a9b7-279790bdf554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:58.335830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc59116-ca38-4d27-a9b7-279790bdf554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:58.336115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc59116-ca38-4d27-a9b7-279790bdf554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:58.336392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc59116-ca38-4d27-a9b7-279790bdf554 admin admin] PATH_INFO: `/v3/users/d4e1190f9c6e47e5924b1de7355a90a9` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:58.337570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc59116-ca38-4d27-a9b7-279790bdf554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4e1190f9c6e47e5924b1de7355a90a9)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:58.339508 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc59116-ca38-4d27-a9b7-279790bdf554 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:58.343166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1d9d5a31-babf-4582-8a68-0ca6a9dbf0b8 None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:58.344368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1d9d5a31-babf-4582-8a68-0ca6a9dbf0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:58.431881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ddc59116-ca38-4d27-a9b7-279790bdf554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4e1190f9c6e47e5924b1de7355a90a9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:58.432619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ddc59116-ca38-4d27-a9b7-279790bdf554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4e1190f9c6e47e5924b1de7355a90a9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:58.433649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1118/2219] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:45:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:45:58.440205 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ddc59116-ca38-4d27-a9b7-279790bdf554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4e1190f9c6e47e5924b1de7355a90a9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:58.461400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ddc59116-ca38-4d27-a9b7-279790bdf554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4e1190f9c6e47e5924b1de7355a90a9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:58.462103 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ddc59116-ca38-4d27-a9b7-279790bdf554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4e1190f9c6e47e5924b1de7355a90a9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:58.485524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d5a31-babf-4582-8a68-0ca6a9dbf0b8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:58.486901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1d9d5a31-babf-4582-8a68-0ca6a9dbf0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:58.487475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1102/2220] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:45:58 2022] DELETE /identity/v3/users/d4e1190f9c6e47e5924b1de7355a90a9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:58.492059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003df94a-cea5-44bc-9977-57f49d5fd2a1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:58.492364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003df94a-cea5-44bc-9977-57f49d5fd2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:58.492689 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003df94a-cea5-44bc-9977-57f49d5fd2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:58.514721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-003df94a-cea5-44bc-9977-57f49d5fd2a1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:58.515975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-003df94a-cea5-44bc-9977-57f49d5fd2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:58.574817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d5a31-babf-4582-8a68-0ca6a9dbf0b8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:58.587663 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1103/2221] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:45:58.601552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003df94a-cea5-44bc-9977-57f49d5fd2a1 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:58.603890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-003df94a-cea5-44bc-9977-57f49d5fd2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:58.691102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd49e02-0f81-4bfd-a548-7b920b2c1256 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:58.693403 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd49e02-0f81-4bfd-a548-7b920b2c1256 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:58.693758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd49e02-0f81-4bfd-a548-7b920b2c1256 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:58.694115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd49e02-0f81-4bfd-a548-7b920b2c1256 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:58.694958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003df94a-cea5-44bc-9977-57f49d5fd2a1 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:58.696885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2bd49e02-0f81-4bfd-a548-7b920b2c1256 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:58.699016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfca53-76cc-48cf-a823-7b3f3f90e16d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:58.699735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfca53-76cc-48cf-a823-7b3f3f90e16d service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:58.700080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfca53-76cc-48cf-a823-7b3f3f90e16d service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:58.700422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfca53-76cc-48cf-a823-7b3f3f90e16d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:58.722793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfca53-76cc-48cf-a823-7b3f3f90e16d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:58.722793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfca53-76cc-48cf-a823-7b3f3f90e16d service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:58.732999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1104/2222] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:45:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:45:58.735844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6bda6-10d0-475b-8104-c8eca76f4fb1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:58.736135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6bda6-10d0-475b-8104-c8eca76f4fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:58.736445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6bda6-10d0-475b-8104-c8eca76f4fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:58.736980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:58.770638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-30a6bda6-10d0-475b-8104-c8eca76f4fb1 None None] MFA Rules not processed for user `007760fc26c64cc9a9a3e6ea4325af31`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:58.772745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-30a6bda6-10d0-475b-8104-c8eca76f4fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:58.798869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd49e02-0f81-4bfd-a548-7b920b2c1256 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:58.800804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd49e02-0f81-4bfd-a548-7b920b2c1256 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:58.802932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:45:58.835592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1119/2223] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:45:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:45:58.866721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1105/2224] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:59.233502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd49e02-0f81-4bfd-a548-7b920b2c1256 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:59.238716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2bd49e02-0f81-4bfd-a548-7b920b2c1256 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:59.262198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd49e02-0f81-4bfd-a548-7b920b2c1256 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:59.278451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:59.279121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:59.279395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:59.279667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] PATH_INFO: `/v3/projects/0d743e4a520b446ba88886cdd6c376fc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:59.280987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d743e4a520b446ba88886cdd6c376fc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:59.282908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:59.409534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d743e4a520b446ba88886cdd6c376fc', 'request_id': 'req-b0e0ac01-26aa-4e93-9c69-aae3d471b236'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:59.426947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d743e4a520b446ba88886cdd6c376fc', 'request_id': 'req-b0e0ac01-26aa-4e93-9c69-aae3d471b236'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:59.434581 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d743e4a520b446ba88886cdd6c376fc', 'request_id': 'req-b0e0ac01-26aa-4e93-9c69-aae3d471b236'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:59.454386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] The token cache is being invalidate because project 0d743e4a520b446ba88886cdd6c376fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:45:59.455529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:59.456921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b0e0ac01-26aa-4e93-9c69-aae3d471b236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:45:59.462380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1120/2225] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:45:59 2022] DELETE /identity/v3/projects/0d743e4a520b446ba88886cdd6c376fc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:45:59.464418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202a3d-2dda-4745-a04f-40a1cf63088d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:59.464859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202a3d-2dda-4745-a04f-40a1cf63088d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:59.465237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202a3d-2dda-4745-a04f-40a1cf63088d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:59.494736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-aa202a3d-2dda-4745-a04f-40a1cf63088d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:59.496743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-aa202a3d-2dda-4745-a04f-40a1cf63088d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:59.642974 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1106/2226] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:59.644537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86548f-2dfb-4c1d-b0fa-79f5668f0b2d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:59.645370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86548f-2dfb-4c1d-b0fa-79f5668f0b2d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:59.645657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86548f-2dfb-4c1d-b0fa-79f5668f0b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:59.680448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9c86548f-2dfb-4c1d-b0fa-79f5668f0b2d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:45:59.681721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c86548f-2dfb-4c1d-b0fa-79f5668f0b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:45:59.694209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1121/2227] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:45:59.695436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa202a3d-2dda-4745-a04f-40a1cf63088d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:59.707732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa202a3d-2dda-4745-a04f-40a1cf63088d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:59.721404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d718c9e-ef17-4be7-a42d-1671ac36a938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:59.722160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d718c9e-ef17-4be7-a42d-1671ac36a938 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:59.723611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d718c9e-ef17-4be7-a42d-1671ac36a938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:59.724147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d718c9e-ef17-4be7-a42d-1671ac36a938 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:59.725054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d718c9e-ef17-4be7-a42d-1671ac36a938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:45:59.725479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d718c9e-ef17-4be7-a42d-1671ac36a938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:59.727821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d718c9e-ef17-4be7-a42d-1671ac36a938 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:59.763448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1107/2228] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:45:59.768346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86548f-2dfb-4c1d-b0fa-79f5668f0b2d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:59.779616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86548f-2dfb-4c1d-b0fa-79f5668f0b2d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:45:59.791353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441aee2-a12c-441a-8564-a2b47e049e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:45:59.791999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441aee2-a12c-441a-8564-a2b47e049e97 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:45:59.792353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441aee2-a12c-441a-8564-a2b47e049e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:45:59.795457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441aee2-a12c-441a-8564-a2b47e049e97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:45:59.796308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441aee2-a12c-441a-8564-a2b47e049e97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:45:59.798369 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441aee2-a12c-441a-8564-a2b47e049e97 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:45:59.867370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1122/2229] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:45:59 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:45:59.871608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d718c9e-ef17-4be7-a42d-1671ac36a938 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:45:59.885518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d718c9e-ef17-4be7-a42d-1671ac36a938 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:00.006661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64e6a-6c1b-4846-9cdc-02ebc0716247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:00.007573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64e6a-6c1b-4846-9cdc-02ebc0716247 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:00.008013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64e6a-6c1b-4846-9cdc-02ebc0716247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:00.009663 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64e6a-6c1b-4846-9cdc-02ebc0716247 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:00.010626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b64e6a-6c1b-4846-9cdc-02ebc0716247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:00.013359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b64e6a-6c1b-4846-9cdc-02ebc0716247 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:00.049075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1108/2230] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:45:59 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:46:00.053510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441aee2-a12c-441a-8564-a2b47e049e97 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:00.074153 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441aee2-a12c-441a-8564-a2b47e049e97 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:00.080888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04e0b8-82e2-4b71-a7d7-1ef70c316322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:00.081972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04e0b8-82e2-4b71-a7d7-1ef70c316322 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:00.083028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04e0b8-82e2-4b71-a7d7-1ef70c316322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:00.083719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04e0b8-82e2-4b71-a7d7-1ef70c316322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:00.084705 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04e0b8-82e2-4b71-a7d7-1ef70c316322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:00.100146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04e0b8-82e2-4b71-a7d7-1ef70c316322 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:00.132717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1123/2231] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:46:00.133653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64e6a-6c1b-4846-9cdc-02ebc0716247 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:00.148312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64e6a-6c1b-4846-9cdc-02ebc0716247 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:00.197288 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e2fc3-0184-4aad-aef1-1b14276b5e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:00.198382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e2fc3-0184-4aad-aef1-1b14276b5e3b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:00.198382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e2fc3-0184-4aad-aef1-1b14276b5e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:00.198871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e2fc3-0184-4aad-aef1-1b14276b5e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:00.199606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e2fc3-0184-4aad-aef1-1b14276b5e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:00.202767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e2fc3-0184-4aad-aef1-1b14276b5e3b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:00.227183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1109/2232] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:46:00.231520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04e0b8-82e2-4b71-a7d7-1ef70c316322 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:00.251043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04e0b8-82e2-4b71-a7d7-1ef70c316322 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:00.266947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da583594-1dec-4b51-bf20-7359dd0eb7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:00.268021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da583594-1dec-4b51-bf20-7359dd0eb7a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:00.268453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da583594-1dec-4b51-bf20-7359dd0eb7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:00.268893 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da583594-1dec-4b51-bf20-7359dd0eb7a3 admin admin] PATH_INFO: `/v3/projects/a3ed54f491dd401597bb4baeea511bff/users/d798d01e81914b9582efee10af70a460/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:00.324213 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da583594-1dec-4b51-bf20-7359dd0eb7a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ed54f491dd401597bb4baeea511bff, user_id=d798d01e81914b9582efee10af70a460, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:00.350157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da583594-1dec-4b51-bf20-7359dd0eb7a3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:00.396266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1124/2233] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:46:00 2022] PUT /identity/v3/projects/a3ed54f491dd401597bb4baeea511bff/users/d798d01e81914b9582efee10af70a460/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:46:00.406440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713912b3-7a32-46b6-bb78-534657fa3753 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:00.406734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713912b3-7a32-46b6-bb78-534657fa3753 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:00.407027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713912b3-7a32-46b6-bb78-534657fa3753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:00.457052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-713912b3-7a32-46b6-bb78-534657fa3753 None None] MFA Rules not processed for user `d798d01e81914b9582efee10af70a460`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:46:00.463548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-713912b3-7a32-46b6-bb78-534657fa3753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:00.481608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:46:00.568789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1110/2234] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:46:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:46:00.592802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da583594-1dec-4b51-bf20-7359dd0eb7a3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:00.594676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-da583594-1dec-4b51-bf20-7359dd0eb7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:00.692301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:46:00.700483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da583594-1dec-4b51-bf20-7359dd0eb7a3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:00.705492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad57be-1832-4b28-a156-7bab9d18e58f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:00.706515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad57be-1832-4b28-a156-7bab9d18e58f service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:00.707057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad57be-1832-4b28-a156-7bab9d18e58f service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:00.708462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad57be-1832-4b28-a156-7bab9d18e58f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:00.722917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad57be-1832-4b28-a156-7bab9d18e58f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:00.725080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad57be-1832-4b28-a156-7bab9d18e58f service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:00.761943 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1125/2235] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:46:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:46:03.155828 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713912b3-7a32-46b6-bb78-534657fa3753 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:03.157402 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-713912b3-7a32-46b6-bb78-534657fa3753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:03.299710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713912b3-7a32-46b6-bb78-534657fa3753 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:03.315151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8d0a9-d161-456d-9a8f-9d0862fea9ec None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:03.315151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8d0a9-d161-456d-9a8f-9d0862fea9ec None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:03.315151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8d0a9-d161-456d-9a8f-9d0862fea9ec None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:03.315151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1126/2236] 10.222.0.60 () {54 vars in 961 bytes} [Wed Aug 31 11:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:46:03.316193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed4270-edd3-4c49-92a8-5b073501ab82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:03.316193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed4270-edd3-4c49-92a8-5b073501ab82 service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:03.316193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed4270-edd3-4c49-92a8-5b073501ab82 service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:03.316193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed4270-edd3-4c49-92a8-5b073501ab82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:03.321973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8aed4270-edd3-4c49-92a8-5b073501ab82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:03.328124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4812bd1-0363-43e1-b75d-084012b6fc99 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:03.328395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4812bd1-0363-43e1-b75d-084012b6fc99 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:03.328690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4812bd1-0363-43e1-b75d-084012b6fc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:03.351513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a4812bd1-0363-43e1-b75d-084012b6fc99 None None] MFA Rules not processed for user `f9114a71a3a247188b263d87b6cb12c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:46:03.352329 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed4270-edd3-4c49-92a8-5b073501ab82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:03.352815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a4812bd1-0363-43e1-b75d-084012b6fc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:03.353990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed4270-edd3-4c49-92a8-5b073501ab82 service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:03.368214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1127/2237] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Aug 31 11:46:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:46:03.372751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1111/2238] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:46:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:46:03.376136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4812bd1-0363-43e1-b75d-084012b6fc99 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:03.390476 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4812bd1-0363-43e1-b75d-084012b6fc99 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:03.406194 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26089329-7d3c-4d2f-8f9a-c75a027d1a36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:03.406830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26089329-7d3c-4d2f-8f9a-c75a027d1a36 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:03.407111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26089329-7d3c-4d2f-8f9a-c75a027d1a36 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:03.407380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26089329-7d3c-4d2f-8f9a-c75a027d1a36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:03.436482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26089329-7d3c-4d2f-8f9a-c75a027d1a36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:03.439308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26089329-7d3c-4d2f-8f9a-c75a027d1a36 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:03.465207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1128/2239] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:46:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:46:08.538889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d97bb2-5b9f-4ddb-905b-d59a95c5981c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:08.542261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d97bb2-5b9f-4ddb-905b-d59a95c5981c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:08.542715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d97bb2-5b9f-4ddb-905b-d59a95c5981c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:08.580359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-03d97bb2-5b9f-4ddb-905b-d59a95c5981c None None] MFA Rules not processed for user `38ea91bf04e741c793e4a06db9fb046f`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:46:08.586144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-03d97bb2-5b9f-4ddb-905b-d59a95c5981c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:08.747349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1112/2240] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:46:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:46:12.757264 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b4ed8-217c-451f-9ab2-c9db5ef5e0a3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:12.757879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b4ed8-217c-451f-9ab2-c9db5ef5e0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:12.757879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b4ed8-217c-451f-9ab2-c9db5ef5e0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:12.778025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-744b4ed8-217c-451f-9ab2-c9db5ef5e0a3 None None] MFA Rules not processed for user `255f7abaa92444309e5bf97f9d91c840`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:46:12.779248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-744b4ed8-217c-451f-9ab2-c9db5ef5e0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:12.851815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1129/2241] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:46:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:46:12.918492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d97bb2-5b9f-4ddb-905b-d59a95c5981c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:12.998331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:46:13.016295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d97bb2-5b9f-4ddb-905b-d59a95c5981c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:13.022404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:13.023709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:13.024859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:13.025439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] PATH_INFO: `/v3/users/d798d01e81914b9582efee10af70a460` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:13.026872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d798d01e81914b9582efee10af70a460)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:13.029742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:13.093886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd798d01e81914b9582efee10af70a460'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:46:13.112160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd798d01e81914b9582efee10af70a460'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:46:13.112160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd798d01e81914b9582efee10af70a460'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:46:13.121983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd798d01e81914b9582efee10af70a460'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:46:13.123248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd798d01e81914b9582efee10af70a460'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:46:13.145275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1113/2242] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:46:12 2022] DELETE /identity/v3/users/d798d01e81914b9582efee10af70a460 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:46:13.150317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78970dd8-d989-46de-9421-50c0e8c65072 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:13.150713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78970dd8-d989-46de-9421-50c0e8c65072 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:13.151088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78970dd8-d989-46de-9421-50c0e8c65072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:13.173378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-78970dd8-d989-46de-9421-50c0e8c65072 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:46:13.174676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-78970dd8-d989-46de-9421-50c0e8c65072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:13.246489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1130/2243] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:46:13.257368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:13.259023 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:13.322927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7661c60-c901-42a9-97df-a2d25f01a0cf admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:13.326805 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f922547-c060-4ba3-b5ff-0a99928b9122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:13.327417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922547-c060-4ba3-b5ff-0a99928b9122 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:13.327708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922547-c060-4ba3-b5ff-0a99928b9122 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:13.327984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922547-c060-4ba3-b5ff-0a99928b9122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:13.338587 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f922547-c060-4ba3-b5ff-0a99928b9122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:13.340073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f922547-c060-4ba3-b5ff-0a99928b9122 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:13.350284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1114/2244] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:46:13.743833 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78970dd8-d989-46de-9421-50c0e8c65072 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:13.745260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-78970dd8-d989-46de-9421-50c0e8c65072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:13.766869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78970dd8-d989-46de-9421-50c0e8c65072 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:13.774429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:13.775672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:13.776101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:13.776767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] PATH_INFO: `/v3/projects/a3ed54f491dd401597bb4baeea511bff` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:13.778296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ed54f491dd401597bb4baeea511bff)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:13.781115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:13.850724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ed54f491dd401597bb4baeea511bff', 'request_id': 'req-67fc46fd-1866-4543-ad57-49e7bfa17ac9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:46:13.861775 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ed54f491dd401597bb4baeea511bff', 'request_id': 'req-67fc46fd-1866-4543-ad57-49e7bfa17ac9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:46:13.873699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ed54f491dd401597bb4baeea511bff', 'request_id': 'req-67fc46fd-1866-4543-ad57-49e7bfa17ac9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:46:13.887082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:46:13.894423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] The token cache is being invalidate because project a3ed54f491dd401597bb4baeea511bff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:46:13.894813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:46:13.895673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-67fc46fd-1866-4543-ad57-49e7bfa17ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:46:13.902363 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1131/2245] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:46:13 2022] DELETE /identity/v3/projects/a3ed54f491dd401597bb4baeea511bff => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:46:13.905000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83c02c-fdae-4eee-a5e2-1f9bba4d74b9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:13.905114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83c02c-fdae-4eee-a5e2-1f9bba4d74b9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:13.905295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83c02c-fdae-4eee-a5e2-1f9bba4d74b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:13.927181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3d83c02c-fdae-4eee-a5e2-1f9bba4d74b9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:46:13.929232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3d83c02c-fdae-4eee-a5e2-1f9bba4d74b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:14.024526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1115/2246] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:46:14.030905 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c176669-c9eb-47f9-8cd4-35f04b5f2fc0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:14.031390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c176669-c9eb-47f9-8cd4-35f04b5f2fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:14.031767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c176669-c9eb-47f9-8cd4-35f04b5f2fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:14.059049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9c176669-c9eb-47f9-8cd4-35f04b5f2fc0 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:46:14.061816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c176669-c9eb-47f9-8cd4-35f04b5f2fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:14.071967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1132/2247] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:46:14.076028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83c02c-fdae-4eee-a5e2-1f9bba4d74b9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:14.089069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83c02c-fdae-4eee-a5e2-1f9bba4d74b9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:14.095249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96bca3-47c1-44a7-8b7f-e3f72f5c0d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:14.096280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96bca3-47c1-44a7-8b7f-e3f72f5c0d86 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:14.096655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96bca3-47c1-44a7-8b7f-e3f72f5c0d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:14.097192 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96bca3-47c1-44a7-8b7f-e3f72f5c0d86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:14.098011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96bca3-47c1-44a7-8b7f-e3f72f5c0d86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:46:14.098473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96bca3-47c1-44a7-8b7f-e3f72f5c0d86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:14.101249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96bca3-47c1-44a7-8b7f-e3f72f5c0d86 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:14.111118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:46:14.116018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1116/2248] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:46:14.124792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c176669-c9eb-47f9-8cd4-35f04b5f2fc0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:14.130696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c176669-c9eb-47f9-8cd4-35f04b5f2fc0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:14.141394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da7ce4-dca6-4c24-9478-21a1e6759e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:14.143168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da7ce4-dca6-4c24-9478-21a1e6759e75 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:14.143869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da7ce4-dca6-4c24-9478-21a1e6759e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:14.144556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da7ce4-dca6-4c24-9478-21a1e6759e75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:14.146448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da7ce4-dca6-4c24-9478-21a1e6759e75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:14.149326 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da7ce4-dca6-4c24-9478-21a1e6759e75 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:14.201243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1133/2249] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:46:14 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:46:14.203441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96bca3-47c1-44a7-8b7f-e3f72f5c0d86 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:14.215829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96bca3-47c1-44a7-8b7f-e3f72f5c0d86 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:14.316755 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b7d2b-4d87-46a6-85e9-9721b8537f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:14.317824 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b7d2b-4d87-46a6-85e9-9721b8537f2c admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:14.318795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b7d2b-4d87-46a6-85e9-9721b8537f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:14.319264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b7d2b-4d87-46a6-85e9-9721b8537f2c admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:14.320420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b7d2b-4d87-46a6-85e9-9721b8537f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:14.326774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b7d2b-4d87-46a6-85e9-9721b8537f2c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:14.356606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1117/2250] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:46:14 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:46:14.358931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da7ce4-dca6-4c24-9478-21a1e6759e75 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:14.370657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da7ce4-dca6-4c24-9478-21a1e6759e75 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:14.376348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3df734-96e6-456b-8402-5fc09e3948f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:14.376997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df734-96e6-456b-8402-5fc09e3948f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:14.377722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df734-96e6-456b-8402-5fc09e3948f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:14.377722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df734-96e6-456b-8402-5fc09e3948f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:14.378184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3df734-96e6-456b-8402-5fc09e3948f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:14.380102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3df734-96e6-456b-8402-5fc09e3948f8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:14.392989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1134/2251] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:46:14.398711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b7d2b-4d87-46a6-85e9-9721b8537f2c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:14.410402 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b7d2b-4d87-46a6-85e9-9721b8537f2c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:14.417176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388864c8-2169-4090-89a6-8fd4d3670316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:14.417895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388864c8-2169-4090-89a6-8fd4d3670316 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:14.418611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388864c8-2169-4090-89a6-8fd4d3670316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:14.418611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388864c8-2169-4090-89a6-8fd4d3670316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:14.419138 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388864c8-2169-4090-89a6-8fd4d3670316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:14.421211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388864c8-2169-4090-89a6-8fd4d3670316 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:14.438134 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1118/2252] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:46:14.442908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3df734-96e6-456b-8402-5fc09e3948f8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:14.455298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3df734-96e6-456b-8402-5fc09e3948f8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:14.462797 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05d53f-b303-4cf2-9ac1-e1c771e4a94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:14.463620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d53f-b303-4cf2-9ac1-e1c771e4a94c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:14.463945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d53f-b303-4cf2-9ac1-e1c771e4a94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:14.464306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d53f-b303-4cf2-9ac1-e1c771e4a94c admin admin] PATH_INFO: `/v3/projects/2351e519c01641e7aa832e41ad030cf8/users/c14dfcf40ed34d0b912599cbafeef8ca/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:14.487005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05d53f-b303-4cf2-9ac1-e1c771e4a94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2351e519c01641e7aa832e41ad030cf8, user_id=c14dfcf40ed34d0b912599cbafeef8ca, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:14.491032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05d53f-b303-4cf2-9ac1-e1c771e4a94c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:14.507023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1135/2253] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:46:14 2022] PUT /identity/v3/projects/2351e519c01641e7aa832e41ad030cf8/users/c14dfcf40ed34d0b912599cbafeef8ca/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:46:14.517325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f13a9-ddc8-4502-b16f-047fb2094003 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:14.517644 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f13a9-ddc8-4502-b16f-047fb2094003 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:14.518083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f13a9-ddc8-4502-b16f-047fb2094003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:14.550435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-094f13a9-ddc8-4502-b16f-047fb2094003 None None] MFA Rules not processed for user `c14dfcf40ed34d0b912599cbafeef8ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:46:14.551862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-094f13a9-ddc8-4502-b16f-047fb2094003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:14.629359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1119/2254] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:46:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:46:14.642594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05d53f-b303-4cf2-9ac1-e1c771e4a94c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:14.644427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ea05d53f-b303-4cf2-9ac1-e1c771e4a94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:14.740807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05d53f-b303-4cf2-9ac1-e1c771e4a94c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:14.744653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67577a6-becc-4ef6-872d-4a04a3c4b676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:14.745353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67577a6-becc-4ef6-872d-4a04a3c4b676 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:14.746026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67577a6-becc-4ef6-872d-4a04a3c4b676 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:14.746375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67577a6-becc-4ef6-872d-4a04a3c4b676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:14.762350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67577a6-becc-4ef6-872d-4a04a3c4b676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:14.765337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67577a6-becc-4ef6-872d-4a04a3c4b676 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:14.783044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1136/2255] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:46:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:46:16.014102 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31b7bc-e650-47e6-82a1-0eb35531bb34 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:16.014602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31b7bc-e650-47e6-82a1-0eb35531bb34 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:16.014706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31b7bc-e650-47e6-82a1-0eb35531bb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:16.035425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4c31b7bc-e650-47e6-82a1-0eb35531bb34 None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:46:16.036713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4c31b7bc-e650-47e6-82a1-0eb35531bb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:16.101373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1120/2256] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:46:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:46:16.114359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67577a6-becc-4ef6-872d-4a04a3c4b676 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:16.117341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f67577a6-becc-4ef6-872d-4a04a3c4b676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:16.209646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67577a6-becc-4ef6-872d-4a04a3c4b676 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:16.216466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af43be-eeb3-403e-be8f-d7e4b37219dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:16.217607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af43be-eeb3-403e-be8f-d7e4b37219dc service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:16.218084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af43be-eeb3-403e-be8f-d7e4b37219dc service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:16.218414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af43be-eeb3-403e-be8f-d7e4b37219dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:16.228926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af43be-eeb3-403e-be8f-d7e4b37219dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:16.230701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af43be-eeb3-403e-be8f-d7e4b37219dc service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:16.241337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1137/2257] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:46:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:46:16.424381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c34fb-7632-49f4-812e-92b3ee0e9875 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:16.424705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c34fb-7632-49f4-812e-92b3ee0e9875 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:16.424985 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c34fb-7632-49f4-812e-92b3ee0e9875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:16.451347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-451c34fb-7632-49f4-812e-92b3ee0e9875 None None] MFA Rules not processed for user `c14dfcf40ed34d0b912599cbafeef8ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:46:16.452796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-451c34fb-7632-49f4-812e-92b3ee0e9875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:16.472152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1121/2258] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:46:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:46:26.235822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:46:26.444847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:47:02.205335 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456dcae-494c-4012-ab98-f7775cac0684 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:02.205922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456dcae-494c-4012-ab98-f7775cac0684 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:02.205922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456dcae-494c-4012-ab98-f7775cac0684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:02.226860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3456dcae-494c-4012-ab98-f7775cac0684 None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:47:02.228065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3456dcae-494c-4012-ab98-f7775cac0684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:02.235314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1138/2259] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:47:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:47:02.268958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c34fb-7632-49f4-812e-92b3ee0e9875 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:02.284688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c34fb-7632-49f4-812e-92b3ee0e9875 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:02.294479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdce08-3f2e-4642-8bc9-a94af3faa29b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:02.295566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdce08-3f2e-4642-8bc9-a94af3faa29b service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:02.295876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdce08-3f2e-4642-8bc9-a94af3faa29b service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:02.296265 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdce08-3f2e-4642-8bc9-a94af3faa29b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:02.309939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdce08-3f2e-4642-8bc9-a94af3faa29b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:02.312736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdce08-3f2e-4642-8bc9-a94af3faa29b service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:02.329916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1122/2260] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:47:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:47:02.948783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb52c6-19ff-4585-b766-25c391cb2f89 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:02.949070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb52c6-19ff-4585-b766-25c391cb2f89 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:02.949369 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb52c6-19ff-4585-b766-25c391cb2f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:02.969198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c2cb52c6-19ff-4585-b766-25c391cb2f89 None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:47:02.970409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c2cb52c6-19ff-4585-b766-25c391cb2f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:02.977593 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1139/2261] 10.222.0.60 () {58 vars in 967 bytes} [Wed Aug 31 11:47:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:47:02.991352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdce08-3f2e-4642-8bc9-a94af3faa29b service swift] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:03.002042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdce08-3f2e-4642-8bc9-a94af3faa29b service swift] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:03.007959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b71dd4-ea78-4329-8dbf-a074f769e6e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:03.008615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b71dd4-ea78-4329-8dbf-a074f769e6e2 service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:03.008892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b71dd4-ea78-4329-8dbf-a074f769e6e2 service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:03.009169 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b71dd4-ea78-4329-8dbf-a074f769e6e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:03.018416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b71dd4-ea78-4329-8dbf-a074f769e6e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:03.019950 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b71dd4-ea78-4329-8dbf-a074f769e6e2 service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:03.028683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1123/2262] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:47:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:47:03.323433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd01f-0875-4f3a-8e1f-6a8330be2e1f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:03.324192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd01f-0875-4f3a-8e1f-6a8330be2e1f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:03.324192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd01f-0875-4f3a-8e1f-6a8330be2e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:03.346330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-03bdd01f-0875-4f3a-8e1f-6a8330be2e1f None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:47:03.348189 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-03bdd01f-0875-4f3a-8e1f-6a8330be2e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:03.357043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1140/2263] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:47:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:47:03.378000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b71dd4-ea78-4329-8dbf-a074f769e6e2 service swift] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:03.398265 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b71dd4-ea78-4329-8dbf-a074f769e6e2 service swift] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:03.404814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfb0e2-8980-4fd3-b025-74f40de76dec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:03.405514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfb0e2-8980-4fd3-b025-74f40de76dec service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:03.405841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfb0e2-8980-4fd3-b025-74f40de76dec service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:03.406161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfb0e2-8980-4fd3-b025-74f40de76dec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:03.416812 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfb0e2-8980-4fd3-b025-74f40de76dec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:03.418526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfb0e2-8980-4fd3-b025-74f40de76dec service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:03.427057 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1124/2264] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:47:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 11:47:03.797383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9963fa8-cc92-4e9b-a4da-b34ee6c1044e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:03.797829 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9963fa8-cc92-4e9b-a4da-b34ee6c1044e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:03.798333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9963fa8-cc92-4e9b-a4da-b34ee6c1044e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:03.819710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c9963fa8-cc92-4e9b-a4da-b34ee6c1044e None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:47:03.820956 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c9963fa8-cc92-4e9b-a4da-b34ee6c1044e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:03.828094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1141/2265] 10.222.0.60 () {58 vars in 967 bytes} [Wed Aug 31 11:47:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:47:03.841352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfb0e2-8980-4fd3-b025-74f40de76dec service swift] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:03.853044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfb0e2-8980-4fd3-b025-74f40de76dec service swift] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:03.858398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75c9be-4bfc-402f-b402-b66f51b603cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:03.859040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75c9be-4bfc-402f-b402-b66f51b603cc service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:03.859338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75c9be-4bfc-402f-b402-b66f51b603cc service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:03.859629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75c9be-4bfc-402f-b402-b66f51b603cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:03.868743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75c9be-4bfc-402f-b402-b66f51b603cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:03.870504 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75c9be-4bfc-402f-b402-b66f51b603cc service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:03.879027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1125/2266] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:47:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:47:06.035428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9963fa8-cc92-4e9b-a4da-b34ee6c1044e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:06.106819 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9963fa8-cc92-4e9b-a4da-b34ee6c1044e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:06.111571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:06.112269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:06.112591 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:06.112895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] PATH_INFO: `/v3/users/c14dfcf40ed34d0b912599cbafeef8ca` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:06.114017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14dfcf40ed34d0b912599cbafeef8ca)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:06.116862 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:06.170569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14dfcf40ed34d0b912599cbafeef8ca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:47:06.178752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14dfcf40ed34d0b912599cbafeef8ca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:47:06.179075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14dfcf40ed34d0b912599cbafeef8ca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:47:06.179685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14dfcf40ed34d0b912599cbafeef8ca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:47:06.187698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14dfcf40ed34d0b912599cbafeef8ca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:47:06.211355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1142/2267] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:47:06 2022] DELETE /identity/v3/users/c14dfcf40ed34d0b912599cbafeef8ca => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:47:06.216000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaaf10-4ea0-4f79-8104-a2c57e09d92a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:06.216528 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaaf10-4ea0-4f79-8104-a2c57e09d92a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:06.216872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaaf10-4ea0-4f79-8104-a2c57e09d92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:06.236633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cfaaaf10-4ea0-4f79-8104-a2c57e09d92a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:47:06.237862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cfaaaf10-4ea0-4f79-8104-a2c57e09d92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:06.298852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1126/2268] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:47:06.309043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:06.310447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:06.385135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16716e2-2aff-418a-a2da-b90dff0d79bd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:06.389051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54753fd-c0b4-4fc4-b9f8-a4f8e8731031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:06.389682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54753fd-c0b4-4fc4-b9f8-a4f8e8731031 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:06.389965 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54753fd-c0b4-4fc4-b9f8-a4f8e8731031 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:06.390243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54753fd-c0b4-4fc4-b9f8-a4f8e8731031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:06.399988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:47:06.402067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54753fd-c0b4-4fc4-b9f8-a4f8e8731031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:06.403771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54753fd-c0b4-4fc4-b9f8-a4f8e8731031 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:06.418461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1143/2269] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:47:06.623512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaaaf10-4ea0-4f79-8104-a2c57e09d92a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:06.625300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cfaaaf10-4ea0-4f79-8104-a2c57e09d92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:06.638521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaaaf10-4ea0-4f79-8104-a2c57e09d92a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:06.643284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:06.643922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:06.644200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:06.644512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] PATH_INFO: `/v3/projects/2351e519c01641e7aa832e41ad030cf8` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:06.645774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2351e519c01641e7aa832e41ad030cf8)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:06.647851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:06.715955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2351e519c01641e7aa832e41ad030cf8', 'request_id': 'req-cb5d7375-de9b-46f4-8011-c87463f4c547'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:47:06.722210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2351e519c01641e7aa832e41ad030cf8', 'request_id': 'req-cb5d7375-de9b-46f4-8011-c87463f4c547'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:47:06.729889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2351e519c01641e7aa832e41ad030cf8', 'request_id': 'req-cb5d7375-de9b-46f4-8011-c87463f4c547'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:47:06.746460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] The token cache is being invalidate because project 2351e519c01641e7aa832e41ad030cf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:47:06.746740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:47:06.747581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:47:06.749380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1127/2270] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:47:06 2022] DELETE /identity/v3/projects/2351e519c01641e7aa832e41ad030cf8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:47:08.226224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119c937-35d7-4f58-9e2d-351a5eabad54 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:08.226929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119c937-35d7-4f58-9e2d-351a5eabad54 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:08.227492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119c937-35d7-4f58-9e2d-351a5eabad54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:08.254097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3119c937-35d7-4f58-9e2d-351a5eabad54 None None] MFA Rules not processed for user `5de832888ffd4a69ae88cac6b632304e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:47:08.256349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3119c937-35d7-4f58-9e2d-351a5eabad54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:08.335993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1144/2271] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:47:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:47:16.742817 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:47:18.327419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=27 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:47:32.864838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:32.866351 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:32.924796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d7375-de9b-46f4-8011-c87463f4c547 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:32.928912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d77d5-91a4-42e6-88df-76f8c4d4273c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:32.929658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d77d5-91a4-42e6-88df-76f8c4d4273c service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:32.929947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d77d5-91a4-42e6-88df-76f8c4d4273c service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:32.930227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d77d5-91a4-42e6-88df-76f8c4d4273c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:32.931994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-142d77d5-91a4-42e6-88df-76f8c4d4273c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:32.997863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d77d5-91a4-42e6-88df-76f8c4d4273c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:32.999669 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d77d5-91a4-42e6-88df-76f8c4d4273c service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:33.010064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1128/2272] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:47:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:47:42.269987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612a4bc-0d7b-45ef-981d-10baae5bdcca None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:42.270738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612a4bc-0d7b-45ef-981d-10baae5bdcca None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:42.270738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612a4bc-0d7b-45ef-981d-10baae5bdcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:42.290906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5612a4bc-0d7b-45ef-981d-10baae5bdcca None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:47:42.293217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5612a4bc-0d7b-45ef-981d-10baae5bdcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:42.358825 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1145/2273] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:47:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:47:42.371668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d77d5-91a4-42e6-88df-76f8c4d4273c service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:42.373159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-142d77d5-91a4-42e6-88df-76f8c4d4273c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:42.444648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d77d5-91a4-42e6-88df-76f8c4d4273c service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:42.449684 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe334081-6ea0-4b30-8166-9224961552ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:42.450290 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe334081-6ea0-4b30-8166-9224961552ba service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:42.450572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe334081-6ea0-4b30-8166-9224961552ba service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:42.450843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe334081-6ea0-4b30-8166-9224961552ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:42.460082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe334081-6ea0-4b30-8166-9224961552ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:42.461766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe334081-6ea0-4b30-8166-9224961552ba service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:42.469821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1129/2274] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:47:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:47:52.352379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=25 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:47:52.466531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=82 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:47:55.928130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612a4bc-0d7b-45ef-981d-10baae5bdcca None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:55.929557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5612a4bc-0d7b-45ef-981d-10baae5bdcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:55.942454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612a4bc-0d7b-45ef-981d-10baae5bdcca None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:55.946488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cb921-869d-41ba-82d4-d60da283a942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:55.947083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cb921-869d-41ba-82d4-d60da283a942 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:55.947365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cb921-869d-41ba-82d4-d60da283a942 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:55.947638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cb921-869d-41ba-82d4-d60da283a942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:55.949376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a72cb921-869d-41ba-82d4-d60da283a942 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:56.021964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cb921-869d-41ba-82d4-d60da283a942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:56.023472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cb921-869d-41ba-82d4-d60da283a942 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:56.035597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1146/2275] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:47:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:47:57.223441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe334081-6ea0-4b30-8166-9224961552ba service swift] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:57.232738 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe334081-6ea0-4b30-8166-9224961552ba service swift] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:57.236304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269e5cc-d5f2-4202-881a-673e8c726258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:57.236930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269e5cc-d5f2-4202-881a-673e8c726258 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:57.237214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269e5cc-d5f2-4202-881a-673e8c726258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:57.237487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269e5cc-d5f2-4202-881a-673e8c726258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:57.239378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3269e5cc-d5f2-4202-881a-673e8c726258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:57.253339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269e5cc-d5f2-4202-881a-673e8c726258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:57.255409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269e5cc-d5f2-4202-881a-673e8c726258 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:57.266298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1130/2276] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:47:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:48:03.971043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9b661-1d82-4b1b-a23a-d56188b936e1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:03.971600 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9b661-1d82-4b1b-a23a-d56188b936e1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:03.971837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9b661-1d82-4b1b-a23a-d56188b936e1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:03.973205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1147/2277] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:48:03.979018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230e351-5a55-48e2-b3dc-fa9b8373270d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:03.979494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230e351-5a55-48e2-b3dc-fa9b8373270d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:03.979789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230e351-5a55-48e2-b3dc-fa9b8373270d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:04.002027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3230e351-5a55-48e2-b3dc-fa9b8373270d None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:48:04.003243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3230e351-5a55-48e2-b3dc-fa9b8373270d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:04.013720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1131/2278] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:48:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:48:04.026207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9b661-1d82-4b1b-a23a-d56188b936e1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:48:04.027886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-80b9b661-1d82-4b1b-a23a-d56188b936e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:04.088443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9b661-1d82-4b1b-a23a-d56188b936e1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:48:04.092231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b676ed-11df-4ac6-9891-1a7ed667d075 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ec048788904bc58245e878a8644091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:48:04.098247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b676ed-11df-4ac6-9891-1a7ed667d075 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:04.098247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b676ed-11df-4ac6-9891-1a7ed667d075 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:04.098247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b676ed-11df-4ac6-9891-1a7ed667d075 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:04.104311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b676ed-11df-4ac6-9891-1a7ed667d075 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:48:04.105748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b676ed-11df-4ac6-9891-1a7ed667d075 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:48:04.116627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1148/2279] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:48:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:48:04.184254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ce20b-c8b0-44cf-8098-3e68d292ab99 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:04.184574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ce20b-c8b0-44cf-8098-3e68d292ab99 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:04.184849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ce20b-c8b0-44cf-8098-3e68d292ab99 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:04.186202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1132/2280] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:48:04.191512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8609-c7fa-447e-a802-137947534eb7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:04.192100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8609-c7fa-447e-a802-137947534eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:04.192411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8609-c7fa-447e-a802-137947534eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:04.213654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fc6e8609-c7fa-447e-a802-137947534eb7 None None] MFA Rules not processed for user `30546ba7f5614f878a9ba322c7c55ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:48:04.214856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fc6e8609-c7fa-447e-a802-137947534eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:04.221339 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1149/2281] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:48:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:48:04.233189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ce20b-c8b0-44cf-8098-3e68d292ab99 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:48:04.244732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ce20b-c8b0-44cf-8098-3e68d292ab99 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:48:04.248470 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a5bb7-9bad-4a06-b0e0-af4fdc9fc0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:48:04.249086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a5bb7-9bad-4a06-b0e0-af4fdc9fc0ba service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:04.249403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a5bb7-9bad-4a06-b0e0-af4fdc9fc0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:04.249676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a5bb7-9bad-4a06-b0e0-af4fdc9fc0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:04.257827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a5bb7-9bad-4a06-b0e0-af4fdc9fc0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:48:04.259262 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a5bb7-9bad-4a06-b0e0-af4fdc9fc0ba service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:48:04.266887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1133/2282] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:48:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:48:04.350927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb74e4-8a93-40d8-ae37-ba0ce4b5ff24 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:04.351209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb74e4-8a93-40d8-ae37-ba0ce4b5ff24 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:04.351486 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb74e4-8a93-40d8-ae37-ba0ce4b5ff24 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:04.352863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1150/2283] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:48:04.357601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a1fd3-1311-452a-a545-968dfe0aac19 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:04.357893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a1fd3-1311-452a-a545-968dfe0aac19 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:04.358171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a1fd3-1311-452a-a545-968dfe0aac19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:04.378180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b50a1fd3-1311-452a-a545-968dfe0aac19 None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:48:04.380107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b50a1fd3-1311-452a-a545-968dfe0aac19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:04.389164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1134/2284] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:48:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:48:04.399625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb74e4-8a93-40d8-ae37-ba0ce4b5ff24 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:48:04.411256 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb74e4-8a93-40d8-ae37-ba0ce4b5ff24 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:48:04.414846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30120d-ee42-4b7a-8139-c73569aadcb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:48:04.415447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30120d-ee42-4b7a-8139-c73569aadcb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:04.415726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30120d-ee42-4b7a-8139-c73569aadcb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:04.415999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30120d-ee42-4b7a-8139-c73569aadcb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:04.426972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30120d-ee42-4b7a-8139-c73569aadcb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:48:04.428397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30120d-ee42-4b7a-8139-c73569aadcb2 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:48:04.438939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1151/2285] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:48:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:48:06.685792 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009ae5-da8d-4709-812c-cee2760f45e6 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:06.687229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009ae5-da8d-4709-812c-cee2760f45e6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:06.687519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009ae5-da8d-4709-812c-cee2760f45e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:06.710186 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-88009ae5-da8d-4709-812c-cee2760f45e6 None None] MFA Rules not processed for user `007760fc26c64cc9a9a3e6ea4325af31`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:48:06.711298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-88009ae5-da8d-4709-812c-cee2760f45e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:06.770238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1135/2286] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:48:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:48:06.865584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30120d-ee42-4b7a-8139-c73569aadcb2 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:48:06.867161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9e30120d-ee42-4b7a-8139-c73569aadcb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:06.938772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30120d-ee42-4b7a-8139-c73569aadcb2 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:48:06.942902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fefeef-2741-41b4-9873-ade934b86335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:48:06.943531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fefeef-2741-41b4-9873-ade934b86335 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:06.943807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fefeef-2741-41b4-9873-ade934b86335 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:06.944072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fefeef-2741-41b4-9873-ade934b86335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:06.945792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c3fefeef-2741-41b4-9873-ade934b86335 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:06.970983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:48:07.013834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fefeef-2741-41b4-9873-ade934b86335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:48:07.015475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fefeef-2741-41b4-9873-ade934b86335 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:48:07.027592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1152/2287] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:48:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:48:09.935938 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8d4fa-5ead-4ef0-8c79-5b7b64b870ce None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:09.936441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8d4fa-5ead-4ef0-8c79-5b7b64b870ce None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:09.936778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8d4fa-5ead-4ef0-8c79-5b7b64b870ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:09.960804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-fba8d4fa-5ead-4ef0-8c79-5b7b64b870ce None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:48:09.962317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fba8d4fa-5ead-4ef0-8c79-5b7b64b870ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:09.970784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1136/2288] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:48:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:48:10.001089 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fefeef-2741-41b4-9873-ade934b86335 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:48:10.014878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fefeef-2741-41b4-9873-ade934b86335 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:48:10.020919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766ad10-66d6-4421-988a-5121445addfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:48:10.021538 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766ad10-66d6-4421-988a-5121445addfc service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:10.021824 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766ad10-66d6-4421-988a-5121445addfc service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:10.022124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766ad10-66d6-4421-988a-5121445addfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:10.032401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766ad10-66d6-4421-988a-5121445addfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:48:10.034067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766ad10-66d6-4421-988a-5121445addfc service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:48:10.043334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1153/2289] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:48:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:48:19.952714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=46 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:48:20.039788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=23 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:48:31.598165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8d4fa-5ead-4ef0-8c79-5b7b64b870ce None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:48:31.609920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8d4fa-5ead-4ef0-8c79-5b7b64b870ce None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:48:31.614589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdf89b-0f6d-406b-99e9-a56ed253a275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:48:31.615293 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf89b-0f6d-406b-99e9-a56ed253a275 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:31.615782 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf89b-0f6d-406b-99e9-a56ed253a275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:31.616231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf89b-0f6d-406b-99e9-a56ed253a275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:31.618016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1cfdf89b-0f6d-406b-99e9-a56ed253a275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:31.627707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdf89b-0f6d-406b-99e9-a56ed253a275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:48:31.629225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdf89b-0f6d-406b-99e9-a56ed253a275 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:48:31.637338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1137/2290] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:48:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:48:41.633734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:48:45.918973 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cff08-1db7-4af0-9dbb-3d84b33829fd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:45.919519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cff08-1db7-4af0-9dbb-3d84b33829fd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:45.919519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cff08-1db7-4af0-9dbb-3d84b33829fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:45.941064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ca6cff08-1db7-4af0-9dbb-3d84b33829fd None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:48:45.942511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ca6cff08-1db7-4af0-9dbb-3d84b33829fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:45.949617 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1154/2291] 10.222.0.60 () {58 vars in 967 bytes} [Wed Aug 31 11:48:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:48:45.962300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdf89b-0f6d-406b-99e9-a56ed253a275 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:48:45.974664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdf89b-0f6d-406b-99e9-a56ed253a275 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:48:45.979606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1f658-dd80-4e76-bf68-1dc97277463c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:48:45.980200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1f658-dd80-4e76-bf68-1dc97277463c service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:45.980478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1f658-dd80-4e76-bf68-1dc97277463c service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:45.980779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1f658-dd80-4e76-bf68-1dc97277463c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:45.989424 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1f658-dd80-4e76-bf68-1dc97277463c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:48:45.990997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1f658-dd80-4e76-bf68-1dc97277463c service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:48:45.998740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1138/2292] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:48:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:48:55.935490 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:48:55.995722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:12.888614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cff08-1db7-4af0-9dbb-3d84b33829fd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:12.891186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ca6cff08-1db7-4af0-9dbb-3d84b33829fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:12.908715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cff08-1db7-4af0-9dbb-3d84b33829fd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:12.921341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ef1b-7858-4871-ae77-9688ad2ea661 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:12.921341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ef1b-7858-4871-ae77-9688ad2ea661 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:12.921707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ef1b-7858-4871-ae77-9688ad2ea661 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:12.921909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ef1b-7858-4871-ae77-9688ad2ea661 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:12.924279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c592ef1b-7858-4871-ae77-9688ad2ea661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:12.936854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592ef1b-7858-4871-ae77-9688ad2ea661 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:12.938497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592ef1b-7858-4871-ae77-9688ad2ea661 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:12.950352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1155/2293] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:22.945861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:38.870437 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1f658-dd80-4e76-bf68-1dc97277463c service swift] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:38.883557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1f658-dd80-4e76-bf68-1dc97277463c service swift] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:38.888943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2913d-fa59-4db4-8123-1de0a03f7921 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:38.889530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2913d-fa59-4db4-8123-1de0a03f7921 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:38.889811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2913d-fa59-4db4-8123-1de0a03f7921 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:38.890077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2913d-fa59-4db4-8123-1de0a03f7921 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:38.892756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b0b2913d-fa59-4db4-8123-1de0a03f7921 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:38.949407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2913d-fa59-4db4-8123-1de0a03f7921 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:38.950996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2913d-fa59-4db4-8123-1de0a03f7921 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:38.960575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1139/2294] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:38 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:39.655744 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ef1b-7858-4871-ae77-9688ad2ea661 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:39.657263 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c592ef1b-7858-4871-ae77-9688ad2ea661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:39.717358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2913d-fa59-4db4-8123-1de0a03f7921 service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:39.719543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b0b2913d-fa59-4db4-8123-1de0a03f7921 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:39.744209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ef1b-7858-4871-ae77-9688ad2ea661 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:39.749374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a89a0-de5d-48ee-a91d-7300dcb6851b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:39.749985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a89a0-de5d-48ee-a91d-7300dcb6851b service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:39.750259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a89a0-de5d-48ee-a91d-7300dcb6851b service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:39.750534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a89a0-de5d-48ee-a91d-7300dcb6851b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:39.752349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-422a89a0-de5d-48ee-a91d-7300dcb6851b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:39.764697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a89a0-de5d-48ee-a91d-7300dcb6851b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:39.766309 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a89a0-de5d-48ee-a91d-7300dcb6851b service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:39.777457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1156/2295] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:39.797211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2913d-fa59-4db4-8123-1de0a03f7921 service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:39.802688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:39.803549 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:39.803832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:39.804107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] PATH_INFO: `/v3/users/007760fc26c64cc9a9a3e6ea4325af31` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:39.805333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=007760fc26c64cc9a9a3e6ea4325af31)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:39.807268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:39.853862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '007760fc26c64cc9a9a3e6ea4325af31'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:39.854516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '007760fc26c64cc9a9a3e6ea4325af31'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:39.861352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '007760fc26c64cc9a9a3e6ea4325af31'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:39.868426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '007760fc26c64cc9a9a3e6ea4325af31'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:39.869074 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '007760fc26c64cc9a9a3e6ea4325af31'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:39.885852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1140/2296] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:39 2022] DELETE /identity/v3/users/007760fc26c64cc9a9a3e6ea4325af31 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:40.056443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a89a0-de5d-48ee-a91d-7300dcb6851b service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:40.068255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a89a0-de5d-48ee-a91d-7300dcb6851b service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:40.130680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:40.131338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:40.131623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:40.131897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] PATH_INFO: `/v3/projects/f8f83820c15b492c9013ff8345668b63` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:40.133047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8f83820c15b492c9013ff8345668b63)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:40.134969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:40.183569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8f83820c15b492c9013ff8345668b63', 'request_id': 'req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:40.190513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8f83820c15b492c9013ff8345668b63', 'request_id': 'req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:40.196409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8f83820c15b492c9013ff8345668b63', 'request_id': 'req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:40.208878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] The token cache is being invalidate because project f8f83820c15b492c9013ff8345668b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:40.209169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:40.209789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:40.211266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1157/2297] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:40 2022] DELETE /identity/v3/projects/f8f83820c15b492c9013ff8345668b63 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:49.827489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:49.829015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:49.851245 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:49.906175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d041a6c-f6cf-4caf-9512-cf617f7b74ce admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:49.969229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f43d3-b6c5-4794-912e-89275ff409e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:49.969796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f43d3-b6c5-4794-912e-89275ff409e9 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:49.970222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f43d3-b6c5-4794-912e-89275ff409e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:49.970510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f43d3-b6c5-4794-912e-89275ff409e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:49.972282 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0f1f43d3-b6c5-4794-912e-89275ff409e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:50.029910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f43d3-b6c5-4794-912e-89275ff409e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:50.031572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f43d3-b6c5-4794-912e-89275ff409e9 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:50.041143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1141/2298] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:50.205844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:00.037326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=76 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:08.449021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:08.450380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:08.466064 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa59dc-ba49-405a-b2b6-b5bd0703a79d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:08.471618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed46f1a-6ad7-4eee-8979-7798ba18beaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:08.472206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46f1a-6ad7-4eee-8979-7798ba18beaf service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:08.472506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46f1a-6ad7-4eee-8979-7798ba18beaf service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:08.472787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46f1a-6ad7-4eee-8979-7798ba18beaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:08.474436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3ed46f1a-6ad7-4eee-8979-7798ba18beaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:08.533683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed46f1a-6ad7-4eee-8979-7798ba18beaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:08.535306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed46f1a-6ad7-4eee-8979-7798ba18beaf service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:08.544265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1158/2299] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:50:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:50:09.504667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3444d56-e4c6-4f48-a2f7-994e8f22dd06 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:09.505183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3444d56-e4c6-4f48-a2f7-994e8f22dd06 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:09.505292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3444d56-e4c6-4f48-a2f7-994e8f22dd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:09.524303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e3444d56-e4c6-4f48-a2f7-994e8f22dd06 None None] MFA Rules not processed for user `38ea91bf04e741c793e4a06db9fb046f`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:50:09.525602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e3444d56-e4c6-4f48-a2f7-994e8f22dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:09.581374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1142/2300] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:50:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:50:11.864005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed46f1a-6ad7-4eee-8979-7798ba18beaf service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:11.865546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3ed46f1a-6ad7-4eee-8979-7798ba18beaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:11.881458 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed46f1a-6ad7-4eee-8979-7798ba18beaf service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:11.887451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab65628-2df5-451c-b4bc-6375f4ec6c46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:11.888385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65628-2df5-451c-b4bc-6375f4ec6c46 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:11.888907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65628-2df5-451c-b4bc-6375f4ec6c46 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:11.889365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65628-2df5-451c-b4bc-6375f4ec6c46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:11.892031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4ab65628-2df5-451c-b4bc-6375f4ec6c46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:11.906135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab65628-2df5-451c-b4bc-6375f4ec6c46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:11.907908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab65628-2df5-451c-b4bc-6375f4ec6c46 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:11.919225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1159/2301] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:50:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:50:19.574539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=25 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:21.919078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=34 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:23.789381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5f1b7-207a-4284-a4bf-6e2c92bd269f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:23.791652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5f1b7-207a-4284-a4bf-6e2c92bd269f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:23.791946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5f1b7-207a-4284-a4bf-6e2c92bd269f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:23.814816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-56d5f1b7-207a-4284-a4bf-6e2c92bd269f None None] MFA Rules not processed for user `bcd4602a40964a81931cdadc92d7f281`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:50:23.816042 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-56d5f1b7-207a-4284-a4bf-6e2c92bd269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:23.881722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1143/2302] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:50:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:50:24.282686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab65628-2df5-451c-b4bc-6375f4ec6c46 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:24.284465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4ab65628-2df5-451c-b4bc-6375f4ec6c46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:24.350633 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab65628-2df5-451c-b4bc-6375f4ec6c46 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:24.354653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4e26e-5d44-4708-97c2-205f90dcdc43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:24.355307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4e26e-5d44-4708-97c2-205f90dcdc43 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:24.355654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4e26e-5d44-4708-97c2-205f90dcdc43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:24.355890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4e26e-5d44-4708-97c2-205f90dcdc43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:24.358108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0bc4e26e-5d44-4708-97c2-205f90dcdc43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:24.368127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4e26e-5d44-4708-97c2-205f90dcdc43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:24.369629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4e26e-5d44-4708-97c2-205f90dcdc43 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:24.378620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1160/2303] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:50:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:50:24.724968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5f1b7-207a-4284-a4bf-6e2c92bd269f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:24.726504 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-56d5f1b7-207a-4284-a4bf-6e2c92bd269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:24.804299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5f1b7-207a-4284-a4bf-6e2c92bd269f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:24.814399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5fafc-44ef-4783-9a34-927ee4eb232a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:24.815014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5fafc-44ef-4783-9a34-927ee4eb232a service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:24.815472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5fafc-44ef-4783-9a34-927ee4eb232a service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:24.815748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5fafc-44ef-4783-9a34-927ee4eb232a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:24.818264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-91c5fafc-44ef-4783-9a34-927ee4eb232a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:24.831594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5fafc-44ef-4783-9a34-927ee4eb232a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:24.833295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5fafc-44ef-4783-9a34-927ee4eb232a service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:24.843399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1144/2304] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:50:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:50:25.330909 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22631b4-ef16-4860-adf7-6df6a8b2542a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:25.331838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22631b4-ef16-4860-adf7-6df6a8b2542a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:25.332400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22631b4-ef16-4860-adf7-6df6a8b2542a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:25.353361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c22631b4-ef16-4860-adf7-6df6a8b2542a None None] MFA Rules not processed for user `5de832888ffd4a69ae88cac6b632304e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:50:25.354563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c22631b4-ef16-4860-adf7-6df6a8b2542a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:25.362297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1161/2305] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:50:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:50:34.839794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=57 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:35.089679 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5fafc-44ef-4783-9a34-927ee4eb232a service placement] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:35.103477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5fafc-44ef-4783-9a34-927ee4eb232a service placement] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:35.112795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544a148-896b-49f6-b05a-0c9841c7279b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:35.113569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544a148-896b-49f6-b05a-0c9841c7279b service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:35.113851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544a148-896b-49f6-b05a-0c9841c7279b service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:35.114130 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544a148-896b-49f6-b05a-0c9841c7279b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:35.116046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1544a148-896b-49f6-b05a-0c9841c7279b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:35.130204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544a148-896b-49f6-b05a-0c9841c7279b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:35.132011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544a148-896b-49f6-b05a-0c9841c7279b service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:35.144520 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1145/2306] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:50:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:50:35.347099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=29 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:45.139019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:04.058390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22631b4-ef16-4860-adf7-6df6a8b2542a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:04.059795 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c22631b4-ef16-4860-adf7-6df6a8b2542a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:04.155284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22631b4-ef16-4860-adf7-6df6a8b2542a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:04.160365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d353fb-798a-4304-8435-4e7307a05327 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:04.160994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d353fb-798a-4304-8435-4e7307a05327 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:04.161277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d353fb-798a-4304-8435-4e7307a05327 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:04.161548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d353fb-798a-4304-8435-4e7307a05327 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:04.163313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-41d353fb-798a-4304-8435-4e7307a05327 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:04.174852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d353fb-798a-4304-8435-4e7307a05327 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:04.176417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d353fb-798a-4304-8435-4e7307a05327 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:04.186024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1162/2307] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:51:11.489029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544a148-896b-49f6-b05a-0c9841c7279b service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:11.555413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544a148-896b-49f6-b05a-0c9841c7279b service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:11.560924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa81181-1719-43e7-8aef-8fa09ffa0267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:11.561559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa81181-1719-43e7-8aef-8fa09ffa0267 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:11.562338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa81181-1719-43e7-8aef-8fa09ffa0267 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:11.562800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa81181-1719-43e7-8aef-8fa09ffa0267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:11.564570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5fa81181-1719-43e7-8aef-8fa09ffa0267 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:11.621875 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa81181-1719-43e7-8aef-8fa09ffa0267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:11.623399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa81181-1719-43e7-8aef-8fa09ffa0267 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:11.632256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1146/2308] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:51:14.182010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=37 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:18.170561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1251c-1b43-492b-b97c-4458261ea209 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:18.171843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1251c-1b43-492b-b97c-4458261ea209 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:18.171843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1251c-1b43-492b-b97c-4458261ea209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:18.191366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8bd1251c-1b43-492b-b97c-4458261ea209 None None] MFA Rules not processed for user `38c4d208994a40fe82310adb18bb3547`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:51:18.192624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8bd1251c-1b43-492b-b97c-4458261ea209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:18.199912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1163/2309] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:51:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:51:18.513311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa81181-1719-43e7-8aef-8fa09ffa0267 service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:18.514921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5fa81181-1719-43e7-8aef-8fa09ffa0267 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:18.550381 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa81181-1719-43e7-8aef-8fa09ffa0267 service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:18.554193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b0ee0-10b2-4e40-812c-eedff2790bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:18.554809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b0ee0-10b2-4e40-812c-eedff2790bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:18.555093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b0ee0-10b2-4e40-812c-eedff2790bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:18.555370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b0ee0-10b2-4e40-812c-eedff2790bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:18.557196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b22b0ee0-10b2-4e40-812c-eedff2790bf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:18.567370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b0ee0-10b2-4e40-812c-eedff2790bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:18.569157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b0ee0-10b2-4e40-812c-eedff2790bf3 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:18.577755 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1147/2310] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:51:21.527784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b8fc9-350e-48bc-ba39-87a2035579e2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:21.528284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b8fc9-350e-48bc-ba39-87a2035579e2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:21.528591 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b8fc9-350e-48bc-ba39-87a2035579e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:21.565572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-437b8fc9-350e-48bc-ba39-87a2035579e2 None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:51:21.566856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-437b8fc9-350e-48bc-ba39-87a2035579e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:21.630256 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1164/2311] 10.222.0.60 () {58 vars in 967 bytes} [Wed Aug 31 11:51:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:51:21.682836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b0ee0-10b2-4e40-812c-eedff2790bf3 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:21.684908 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b22b0ee0-10b2-4e40-812c-eedff2790bf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:21.768235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b0ee0-10b2-4e40-812c-eedff2790bf3 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:21.773394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824286a-b5c8-4971-8252-aee920ffcbb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:21.774021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824286a-b5c8-4971-8252-aee920ffcbb8 service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:21.774327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824286a-b5c8-4971-8252-aee920ffcbb8 service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:21.774657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824286a-b5c8-4971-8252-aee920ffcbb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:21.785698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824286a-b5c8-4971-8252-aee920ffcbb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:21.787331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824286a-b5c8-4971-8252-aee920ffcbb8 service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:21.796705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1148/2312] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:51:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:51:22.185111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b8fc9-350e-48bc-ba39-87a2035579e2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:22.195522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b8fc9-350e-48bc-ba39-87a2035579e2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:22.199417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247acaf5-6a3e-4bb0-aef2-1ce9fcd431a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:22.200055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247acaf5-6a3e-4bb0-aef2-1ce9fcd431a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:22.200340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247acaf5-6a3e-4bb0-aef2-1ce9fcd431a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:22.200819 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247acaf5-6a3e-4bb0-aef2-1ce9fcd431a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:22.203416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-247acaf5-6a3e-4bb0-aef2-1ce9fcd431a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:22.313178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247acaf5-6a3e-4bb0-aef2-1ce9fcd431a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:22.314633 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247acaf5-6a3e-4bb0-aef2-1ce9fcd431a2 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:22.337707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1165/2313] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:51:29.746103 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824286a-b5c8-4971-8252-aee920ffcbb8 service swift] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:29.747849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5824286a-b5c8-4971-8252-aee920ffcbb8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:29.757453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:29.768708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824286a-b5c8-4971-8252-aee920ffcbb8 service swift] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:29.782680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:29.783555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:29.783849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:29.784129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] PATH_INFO: `/v3/users/255f7abaa92444309e5bf97f9d91c840` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:29.785435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=255f7abaa92444309e5bf97f9d91c840)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:29.787752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:29.840217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '255f7abaa92444309e5bf97f9d91c840'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:29.840886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '255f7abaa92444309e5bf97f9d91c840'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:29.851128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '255f7abaa92444309e5bf97f9d91c840'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:29.859070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '255f7abaa92444309e5bf97f9d91c840'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:29.859854 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '255f7abaa92444309e5bf97f9d91c840'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:29.878063 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1149/2314] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:51:29 2022] DELETE /identity/v3/users/255f7abaa92444309e5bf97f9d91c840 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:51:30.144000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247acaf5-6a3e-4bb0-aef2-1ce9fcd431a2 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:30.155736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247acaf5-6a3e-4bb0-aef2-1ce9fcd431a2 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:30.213050 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:30.220882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:30.221520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:30.221801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:30.222076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] PATH_INFO: `/v3/projects/e4e02b8409eb43e1bd1d68c79e985430` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:30.223190 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4e02b8409eb43e1bd1d68c79e985430)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:30.225340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:30.275051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4e02b8409eb43e1bd1d68c79e985430', 'request_id': 'req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.282090 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4e02b8409eb43e1bd1d68c79e985430', 'request_id': 'req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.287942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4e02b8409eb43e1bd1d68c79e985430', 'request_id': 'req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.302355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] The token cache is being invalidate because project e4e02b8409eb43e1bd1d68c79e985430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:51:30.302678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.303342 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.304891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1166/2315] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:51:30 2022] DELETE /identity/v3/projects/e4e02b8409eb43e1bd1d68c79e985430 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:51:30.308916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:30.310362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:30.323933 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466517c-4d8d-43f4-bf47-d4991facdb3a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:30.380131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:30.380869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:30.381194 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:30.381531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] PATH_INFO: `/v3/users/eb61e69d574c494da009a6925893213a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:30.382810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb61e69d574c494da009a6925893213a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:30.384973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:30.434088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb61e69d574c494da009a6925893213a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.434832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb61e69d574c494da009a6925893213a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.443040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb61e69d574c494da009a6925893213a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.452858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb61e69d574c494da009a6925893213a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.453802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb61e69d574c494da009a6925893213a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.465116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:30.475899 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1150/2316] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:51:30 2022] DELETE /identity/v3/users/eb61e69d574c494da009a6925893213a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:51:30.656602 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:30.658141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:30.673666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ad92a-2f84-4a2b-badc-b81d83a1e738 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:30.741943 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:30.742568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:30.742854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:30.743127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] PATH_INFO: `/v3/projects/9bd9bcd609ef4b968aedb45b7052a897` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:30.744246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd9bcd609ef4b968aedb45b7052a897)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:30.746196 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:30.792714 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd9bcd609ef4b968aedb45b7052a897', 'request_id': 'req-657b879a-208c-42a0-a49c-abbf0d7b2e75'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.799410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd9bcd609ef4b968aedb45b7052a897', 'request_id': 'req-657b879a-208c-42a0-a49c-abbf0d7b2e75'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.805436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd9bcd609ef4b968aedb45b7052a897', 'request_id': 'req-657b879a-208c-42a0-a49c-abbf0d7b2e75'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.818282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] The token cache is being invalidate because project 9bd9bcd609ef4b968aedb45b7052a897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:51:30.818830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.819465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:30.820999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1167/2317] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:51:30 2022] DELETE /identity/v3/projects/9bd9bcd609ef4b968aedb45b7052a897 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:51:40.475213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:40.815226 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=77,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:41.569309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:41.570848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:41.630519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f7c8c-5fc3-4ef1-8f4a-4024f2376377 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:41.682176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63665561-4250-41d3-9e10-a4fac688d11f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:41.682458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665561-4250-41d3-9e10-a4fac688d11f service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:41.682687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665561-4250-41d3-9e10-a4fac688d11f service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:41.683007 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665561-4250-41d3-9e10-a4fac688d11f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:41.685208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-63665561-4250-41d3-9e10-a4fac688d11f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:41.751100 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63665561-4250-41d3-9e10-a4fac688d11f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:41.752573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63665561-4250-41d3-9e10-a4fac688d11f service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:41.762036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1151/2318] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:51:45.106087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:45.107543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:45.176062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63665561-4250-41d3-9e10-a4fac688d11f service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:45.177930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-63665561-4250-41d3-9e10-a4fac688d11f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:45.191373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63665561-4250-41d3-9e10-a4fac688d11f service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:45.196766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b0c20-49fa-49e0-89cf-e64ff4c855ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:45.197743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b0c20-49fa-49e0-89cf-e64ff4c855ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:45.198047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b0c20-49fa-49e0-89cf-e64ff4c855ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:45.198307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b0c20-49fa-49e0-89cf-e64ff4c855ee admin admin] PATH_INFO: `/v3/users/38ea91bf04e741c793e4a06db9fb046f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:45.199485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b0c20-49fa-49e0-89cf-e64ff4c855ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=38ea91bf04e741c793e4a06db9fb046f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:45.202208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b0c20-49fa-49e0-89cf-e64ff4c855ee admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:45.206519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b879a-208c-42a0-a49c-abbf0d7b2e75 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:45.212800 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5993a-629f-4675-a57a-3f8d200979ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:45.213426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5993a-629f-4675-a57a-3f8d200979ff service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:45.213701 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5993a-629f-4675-a57a-3f8d200979ff service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:45.213972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5993a-629f-4675-a57a-3f8d200979ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:45.215806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a9b5993a-629f-4675-a57a-3f8d200979ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:45.228779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5993a-629f-4675-a57a-3f8d200979ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:45.230399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5993a-629f-4675-a57a-3f8d200979ff service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:45.243179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1168/2319] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:51:45.259358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-331b0c20-49fa-49e0-89cf-e64ff4c855ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38ea91bf04e741c793e4a06db9fb046f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:45.260034 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-331b0c20-49fa-49e0-89cf-e64ff4c855ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38ea91bf04e741c793e4a06db9fb046f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:45.271446 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-331b0c20-49fa-49e0-89cf-e64ff4c855ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38ea91bf04e741c793e4a06db9fb046f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:45.276243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:45.282858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-331b0c20-49fa-49e0-89cf-e64ff4c855ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38ea91bf04e741c793e4a06db9fb046f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:45.283729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-331b0c20-49fa-49e0-89cf-e64ff4c855ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38ea91bf04e741c793e4a06db9fb046f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:45.308591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1152/2320] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:51:45 2022] DELETE /identity/v3/users/38ea91bf04e741c793e4a06db9fb046f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:51:45.526897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5993a-629f-4675-a57a-3f8d200979ff service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:45.538438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5993a-629f-4675-a57a-3f8d200979ff service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:45.612157 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:45.612816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:45.613098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:45.613395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] PATH_INFO: `/v3/projects/0e110109075848a89e6ae1d516de9784` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:45.614515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e110109075848a89e6ae1d516de9784)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:45.616424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:45.666034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e110109075848a89e6ae1d516de9784', 'request_id': 'req-947ce91d-5f28-411f-84fb-5cbab7ec331d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:45.672924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e110109075848a89e6ae1d516de9784', 'request_id': 'req-947ce91d-5f28-411f-84fb-5cbab7ec331d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:45.679627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e110109075848a89e6ae1d516de9784', 'request_id': 'req-947ce91d-5f28-411f-84fb-5cbab7ec331d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:45.694485 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] The token cache is being invalidate because project 0e110109075848a89e6ae1d516de9784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:51:45.694774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:45.695397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-947ce91d-5f28-411f-84fb-5cbab7ec331d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:51:45.696949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1169/2321] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:51:45 2022] DELETE /identity/v3/projects/0e110109075848a89e6ae1d516de9784 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:51:45.702006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58b6d6-f438-4fe6-b49f-3e44e45a5b34 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:45.702382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58b6d6-f438-4fe6-b49f-3e44e45a5b34 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:45.702661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58b6d6-f438-4fe6-b49f-3e44e45a5b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:45.762741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0e58b6d6-f438-4fe6-b49f-3e44e45a5b34 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:51:45.763913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0e58b6d6-f438-4fe6-b49f-3e44e45a5b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:45.832824 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1153/2322] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:51:45.842450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbb1c1-ed8f-4e45-acbe-05d5462eaef0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:45.842859 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbb1c1-ed8f-4e45-acbe-05d5462eaef0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:45.843437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbb1c1-ed8f-4e45-acbe-05d5462eaef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:45.864011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5bbbb1c1-ed8f-4e45-acbe-05d5462eaef0 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:51:45.865279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5bbbb1c1-ed8f-4e45-acbe-05d5462eaef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:45.874106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1170/2323] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:51:45.878129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58b6d6-f438-4fe6-b49f-3e44e45a5b34 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:45.887831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58b6d6-f438-4fe6-b49f-3e44e45a5b34 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:45.892875 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8168e8-af4a-43d1-9b64-cf5ac3d6475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:45.893482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8168e8-af4a-43d1-9b64-cf5ac3d6475d admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:45.893769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8168e8-af4a-43d1-9b64-cf5ac3d6475d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:45.894044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8168e8-af4a-43d1-9b64-cf5ac3d6475d admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:45.894701 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8168e8-af4a-43d1-9b64-cf5ac3d6475d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:51:45.895008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8168e8-af4a-43d1-9b64-cf5ac3d6475d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:45.896654 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8168e8-af4a-43d1-9b64-cf5ac3d6475d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:45.908913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1154/2324] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:51:45.913478 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbb1c1-ed8f-4e45-acbe-05d5462eaef0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:45.923807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbb1c1-ed8f-4e45-acbe-05d5462eaef0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:45.929555 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81849-e8f7-4934-bca6-77f584595bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:45.930172 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81849-e8f7-4934-bca6-77f584595bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:45.930455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81849-e8f7-4934-bca6-77f584595bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:45.930731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81849-e8f7-4934-bca6-77f584595bf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:45.931377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd81849-e8f7-4934-bca6-77f584595bf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:45.933517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd81849-e8f7-4934-bca6-77f584595bf6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:45.947067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:45.965295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1171/2325] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:51:45 2022] POST /identity/v3/projects => generated 357 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:51:45.969932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8168e8-af4a-43d1-9b64-cf5ac3d6475d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:45.979936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8168e8-af4a-43d1-9b64-cf5ac3d6475d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:46.041316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d8e6a-f020-4357-a97f-f1c863bc8be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:46.041838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d8e6a-f020-4357-a97f-f1c863bc8be4 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.042123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d8e6a-f020-4357-a97f-f1c863bc8be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.042403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d8e6a-f020-4357-a97f-f1c863bc8be4 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.043060 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d8e6a-f020-4357-a97f-f1c863bc8be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:46.045156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d8e6a-f020-4357-a97f-f1c863bc8be4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:46.066798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1155/2326] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:51:45 2022] POST /identity/v3/users => generated 279 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:51:46.071537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81849-e8f7-4934-bca6-77f584595bf6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:46.082472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81849-e8f7-4934-bca6-77f584595bf6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:46.087235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6841d2-4458-4295-9eac-12d775a17d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:46.087844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6841d2-4458-4295-9eac-12d775a17d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.088128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6841d2-4458-4295-9eac-12d775a17d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.088403 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6841d2-4458-4295-9eac-12d775a17d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.089113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6841d2-4458-4295-9eac-12d775a17d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:46.090834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6841d2-4458-4295-9eac-12d775a17d7d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:46.101912 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1172/2327] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:51:46.106228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d8e6a-f020-4357-a97f-f1c863bc8be4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:46.117132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d8e6a-f020-4357-a97f-f1c863bc8be4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:46.121829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664378c1-9e42-4f51-ac07-209dfd6395b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:46.122439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664378c1-9e42-4f51-ac07-209dfd6395b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.122726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664378c1-9e42-4f51-ac07-209dfd6395b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.123007 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664378c1-9e42-4f51-ac07-209dfd6395b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.123849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664378c1-9e42-4f51-ac07-209dfd6395b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:46.125509 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664378c1-9e42-4f51-ac07-209dfd6395b0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:46.138085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1156/2328] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:51:46.142685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6841d2-4458-4295-9eac-12d775a17d7d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:46.152929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6841d2-4458-4295-9eac-12d775a17d7d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:46.158623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36eb5b-0fd1-4aa0-8f5b-091b50dff5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:46.159292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36eb5b-0fd1-4aa0-8f5b-091b50dff5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.159571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36eb5b-0fd1-4aa0-8f5b-091b50dff5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.159847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36eb5b-0fd1-4aa0-8f5b-091b50dff5dc admin admin] PATH_INFO: `/v3/projects/ed2af5c1587f4d308877df392c056f3a/users/127e58da6c03461f8df280a6b6495ab5/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.176475 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36eb5b-0fd1-4aa0-8f5b-091b50dff5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2af5c1587f4d308877df392c056f3a, user_id=127e58da6c03461f8df280a6b6495ab5, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:46.179239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36eb5b-0fd1-4aa0-8f5b-091b50dff5dc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:46.194579 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1173/2329] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:51:46 2022] PUT /identity/v3/projects/ed2af5c1587f4d308877df392c056f3a/users/127e58da6c03461f8df280a6b6495ab5/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:51:46.206379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c1b4b-c679-4c1f-af6a-d471b5a227e8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.206907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c1b4b-c679-4c1f-af6a-d471b5a227e8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.207031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c1b4b-c679-4c1f-af6a-d471b5a227e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.240150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:46.243287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3b8c1b4b-c679-4c1f-af6a-d471b5a227e8 None None] MFA Rules not processed for user `127e58da6c03461f8df280a6b6495ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:51:46.245378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3b8c1b4b-c679-4c1f-af6a-d471b5a227e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:46.331429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1157/2330] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:51:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:51:46.335941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36eb5b-0fd1-4aa0-8f5b-091b50dff5dc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:46.405258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36eb5b-0fd1-4aa0-8f5b-091b50dff5dc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:46.410249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3de9d7-dda0-4cdb-8ac9-11a170225a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:46.410869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de9d7-dda0-4cdb-8ac9-11a170225a44 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.411164 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de9d7-dda0-4cdb-8ac9-11a170225a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.411441 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de9d7-dda0-4cdb-8ac9-11a170225a44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.412086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3de9d7-dda0-4cdb-8ac9-11a170225a44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:46.414126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3de9d7-dda0-4cdb-8ac9-11a170225a44 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:46.444152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1174/2331] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:51:46 2022] POST /identity/v3/projects => generated 357 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:51:46.448425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c1b4b-c679-4c1f-af6a-d471b5a227e8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:46.459890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c1b4b-c679-4c1f-af6a-d471b5a227e8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:46.528944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb7420-18fa-48b5-aa14-2cf60c61d6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:46.529610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb7420-18fa-48b5-aa14-2cf60c61d6bc admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.529914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb7420-18fa-48b5-aa14-2cf60c61d6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.530173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb7420-18fa-48b5-aa14-2cf60c61d6bc admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.530851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb7420-18fa-48b5-aa14-2cf60c61d6bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:46.532909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb7420-18fa-48b5-aa14-2cf60c61d6bc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:46.562323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1158/2332] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:51:46 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:51:46.566918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3de9d7-dda0-4cdb-8ac9-11a170225a44 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:46.578618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3de9d7-dda0-4cdb-8ac9-11a170225a44 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:46.585919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689148da-dae2-4765-857f-0425c679fb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:46.586513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689148da-dae2-4765-857f-0425c679fb72 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.586814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689148da-dae2-4765-857f-0425c679fb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.587106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689148da-dae2-4765-857f-0425c679fb72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.588356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689148da-dae2-4765-857f-0425c679fb72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:46.589642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689148da-dae2-4765-857f-0425c679fb72 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:46.602128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1175/2333] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:51:46.606901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb7420-18fa-48b5-aa14-2cf60c61d6bc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:46.623049 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb7420-18fa-48b5-aa14-2cf60c61d6bc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:46.630206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6410f-ed62-4360-b643-c424caa0e284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:46.631128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6410f-ed62-4360-b643-c424caa0e284 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.631423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6410f-ed62-4360-b643-c424caa0e284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.631851 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6410f-ed62-4360-b643-c424caa0e284 admin admin] PATH_INFO: `/v3/projects/2e6dfe12f47b4026bf4c66e5dcf72e28/users/a9371c2629dc4c468ab1c92ff5565b30/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.651014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6410f-ed62-4360-b643-c424caa0e284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6dfe12f47b4026bf4c66e5dcf72e28, user_id=a9371c2629dc4c468ab1c92ff5565b30, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:46.653815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6410f-ed62-4360-b643-c424caa0e284 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:46.668289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1159/2334] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:51:46 2022] PUT /identity/v3/projects/2e6dfe12f47b4026bf4c66e5dcf72e28/users/a9371c2629dc4c468ab1c92ff5565b30/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:51:46.672646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689148da-dae2-4765-857f-0425c679fb72 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:46.683502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689148da-dae2-4765-857f-0425c679fb72 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:46.752424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26391905-5e8a-4175-be89-74d9420467be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:46.753076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26391905-5e8a-4175-be89-74d9420467be admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.753357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26391905-5e8a-4175-be89-74d9420467be admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.753632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26391905-5e8a-4175-be89-74d9420467be admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.754279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26391905-5e8a-4175-be89-74d9420467be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:46.755883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26391905-5e8a-4175-be89-74d9420467be admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:46.767561 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1176/2335] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:51:46.771948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6410f-ed62-4360-b643-c424caa0e284 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:46.783924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6410f-ed62-4360-b643-c424caa0e284 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:46.793059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444db87-af07-48b7-a4ff-f960aaaf08c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:46.793691 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444db87-af07-48b7-a4ff-f960aaaf08c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.793978 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444db87-af07-48b7-a4ff-f960aaaf08c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.794297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444db87-af07-48b7-a4ff-f960aaaf08c9 admin admin] PATH_INFO: `/v3/projects/2e6dfe12f47b4026bf4c66e5dcf72e28/users/a9371c2629dc4c468ab1c92ff5565b30/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.797469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444db87-af07-48b7-a4ff-f960aaaf08c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6dfe12f47b4026bf4c66e5dcf72e28, user_id=a9371c2629dc4c468ab1c92ff5565b30, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:46.800326 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444db87-af07-48b7-a4ff-f960aaaf08c9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:46.812527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1160/2336] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:51:46 2022] PUT /identity/v3/projects/2e6dfe12f47b4026bf4c66e5dcf72e28/users/a9371c2629dc4c468ab1c92ff5565b30/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:51:46.823312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385d4ba-f561-4510-b25c-d35027768059 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:46.823592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385d4ba-f561-4510-b25c-d35027768059 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:46.823859 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385d4ba-f561-4510-b25c-d35027768059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:46.852409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b385d4ba-f561-4510-b25c-d35027768059 None None] MFA Rules not processed for user `a9371c2629dc4c468ab1c92ff5565b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:51:46.853572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b385d4ba-f561-4510-b25c-d35027768059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:46.868077 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:46.929130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1177/2337] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:51:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:51:46.941510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444db87-af07-48b7-a4ff-f960aaaf08c9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:46.943404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d444db87-af07-48b7-a4ff-f960aaaf08c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:47.018898 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:47.041637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444db87-af07-48b7-a4ff-f960aaaf08c9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:47.046335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160c278-ab46-4313-86b7-6aeb87544e61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:47.047179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160c278-ab46-4313-86b7-6aeb87544e61 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:47.047393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160c278-ab46-4313-86b7-6aeb87544e61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:47.047673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160c278-ab46-4313-86b7-6aeb87544e61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:47.057404 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160c278-ab46-4313-86b7-6aeb87544e61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:47.058860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160c278-ab46-4313-86b7-6aeb87544e61 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:47.070185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1161/2338] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:51:47.149629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385d4ba-f561-4510-b25c-d35027768059 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:47.164268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385d4ba-f561-4510-b25c-d35027768059 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:47.168037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e5b67-cdab-4303-8a50-52136196937a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:47.168681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e5b67-cdab-4303-8a50-52136196937a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:47.168967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e5b67-cdab-4303-8a50-52136196937a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:47.169243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e5b67-cdab-4303-8a50-52136196937a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:47.240127 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e5b67-cdab-4303-8a50-52136196937a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:47.241862 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e5b67-cdab-4303-8a50-52136196937a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:47.254802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1178/2339] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:51:57.065821 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=10 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:57.247998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=50 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:02.549397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160c278-ab46-4313-86b7-6aeb87544e61 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:02.618052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160c278-ab46-4313-86b7-6aeb87544e61 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:02.622707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:02.623614 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:02.623990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:02.624269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] PATH_INFO: `/v3/users/127e58da6c03461f8df280a6b6495ab5` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:02.625451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] RBAC: Authorizing `identity:delete_user(user_id=127e58da6c03461f8df280a6b6495ab5)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:02.627407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:02.685114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127e58da6c03461f8df280a6b6495ab5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.686595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127e58da6c03461f8df280a6b6495ab5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.695189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127e58da6c03461f8df280a6b6495ab5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.713635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127e58da6c03461f8df280a6b6495ab5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.714373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127e58da6c03461f8df280a6b6495ab5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.735444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1162/2340] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:52:02 2022] DELETE /identity/v3/users/127e58da6c03461f8df280a6b6495ab5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:02.740271 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b99e7-ccbf-4272-92de-ea85d85103d1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:02.740841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b99e7-ccbf-4272-92de-ea85d85103d1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:02.740943 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b99e7-ccbf-4272-92de-ea85d85103d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:02.761380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cd2b99e7-ccbf-4272-92de-ea85d85103d1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:02.762608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cd2b99e7-ccbf-4272-92de-ea85d85103d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:02.838236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1179/2341] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:52:02.849419 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:02.851592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:02.913901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d979f9-2741-4e13-839f-faf6a735892f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:02.917788 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994ccc6-cc63-4cdb-95d1-ee0b63e36782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:02.918407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994ccc6-cc63-4cdb-95d1-ee0b63e36782 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:02.918687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994ccc6-cc63-4cdb-95d1-ee0b63e36782 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:02.918960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994ccc6-cc63-4cdb-95d1-ee0b63e36782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:02.930273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994ccc6-cc63-4cdb-95d1-ee0b63e36782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:02.931745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994ccc6-cc63-4cdb-95d1-ee0b63e36782 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:02.942642 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1163/2342] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:52:03.251381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b99e7-ccbf-4272-92de-ea85d85103d1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:03.253032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cd2b99e7-ccbf-4272-92de-ea85d85103d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:03.278321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b99e7-ccbf-4272-92de-ea85d85103d1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:03.285720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:03.286589 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:03.286949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:03.287293 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] PATH_INFO: `/v3/projects/ed2af5c1587f4d308877df392c056f3a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:03.288539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2af5c1587f4d308877df392c056f3a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:03.290546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:03.346463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2af5c1587f4d308877df392c056f3a', 'request_id': 'req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:03.354371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2af5c1587f4d308877df392c056f3a', 'request_id': 'req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:03.361321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2af5c1587f4d308877df392c056f3a', 'request_id': 'req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:03.375882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] The token cache is being invalidate because project ed2af5c1587f4d308877df392c056f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:52:03.376160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:03.376843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:03.378567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1180/2343] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:52:03 2022] DELETE /identity/v3/projects/ed2af5c1587f4d308877df392c056f3a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:03.382491 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994ccc6-cc63-4cdb-95d1-ee0b63e36782 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:03.392717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994ccc6-cc63-4cdb-95d1-ee0b63e36782 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:03.412625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:03.455191 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dbb09-e67c-4d1d-9c5e-f8e47a0e3954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:03.455843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dbb09-e67c-4d1d-9c5e-f8e47a0e3954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:03.456127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dbb09-e67c-4d1d-9c5e-f8e47a0e3954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:03.456407 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dbb09-e67c-4d1d-9c5e-f8e47a0e3954 admin admin] PATH_INFO: `/v3/users/a9371c2629dc4c468ab1c92ff5565b30` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:03.457656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7dbb09-e67c-4d1d-9c5e-f8e47a0e3954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9371c2629dc4c468ab1c92ff5565b30)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:03.459655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7dbb09-e67c-4d1d-9c5e-f8e47a0e3954 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:03.509859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2c7dbb09-e67c-4d1d-9c5e-f8e47a0e3954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9371c2629dc4c468ab1c92ff5565b30'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:03.510515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2c7dbb09-e67c-4d1d-9c5e-f8e47a0e3954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9371c2629dc4c468ab1c92ff5565b30'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:03.517723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2c7dbb09-e67c-4d1d-9c5e-f8e47a0e3954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9371c2629dc4c468ab1c92ff5565b30'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:03.533920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2c7dbb09-e67c-4d1d-9c5e-f8e47a0e3954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9371c2629dc4c468ab1c92ff5565b30'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:03.534538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2c7dbb09-e67c-4d1d-9c5e-f8e47a0e3954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9371c2629dc4c468ab1c92ff5565b30'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:03.554015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1164/2344] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:52:03 2022] DELETE /identity/v3/users/a9371c2629dc4c468ab1c92ff5565b30 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:03.884861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:03.887000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:03.908207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01eb93d-cdd2-4d36-af1c-2a5eec0c8c04 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:03.989034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:03.989748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:03.990116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:03.990397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] PATH_INFO: `/v3/projects/2e6dfe12f47b4026bf4c66e5dcf72e28` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:03.991652 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6dfe12f47b4026bf4c66e5dcf72e28)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:03.993660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:04.031314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:04.054521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6dfe12f47b4026bf4c66e5dcf72e28', 'request_id': 'req-fae58882-eabe-4899-bf46-5a97ef61e64c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:04.062315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6dfe12f47b4026bf4c66e5dcf72e28', 'request_id': 'req-fae58882-eabe-4899-bf46-5a97ef61e64c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:04.069282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6dfe12f47b4026bf4c66e5dcf72e28', 'request_id': 'req-fae58882-eabe-4899-bf46-5a97ef61e64c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:04.082771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] The token cache is being invalidate because project 2e6dfe12f47b4026bf4c66e5dcf72e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:52:04.083066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:04.083700 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fae58882-eabe-4899-bf46-5a97ef61e64c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:04.085253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1181/2345] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:52:03 2022] DELETE /identity/v3/projects/2e6dfe12f47b4026bf4c66e5dcf72e28 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:04.091354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f4be3-99e6-4465-8938-4893c4d01eb5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.092220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f4be3-99e6-4465-8938-4893c4d01eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.092740 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f4be3-99e6-4465-8938-4893c4d01eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.113046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cd1f4be3-99e6-4465-8938-4893c4d01eb5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:04.114267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cd1f4be3-99e6-4465-8938-4893c4d01eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:04.175368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1165/2346] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:04.184955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5b202-ac3a-4599-abbc-470085b5ea12 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.185399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5b202-ac3a-4599-abbc-470085b5ea12 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.186340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5b202-ac3a-4599-abbc-470085b5ea12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.207897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-62c5b202-ac3a-4599-abbc-470085b5ea12 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:04.209198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-62c5b202-ac3a-4599-abbc-470085b5ea12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:04.218011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1182/2347] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:04.222345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f4be3-99e6-4465-8938-4893c4d01eb5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:04.234099 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f4be3-99e6-4465-8938-4893c4d01eb5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:04.239413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae8d86-caf5-44be-8d1e-19ffbb7516af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:04.240036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae8d86-caf5-44be-8d1e-19ffbb7516af admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.240327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae8d86-caf5-44be-8d1e-19ffbb7516af admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.240775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae8d86-caf5-44be-8d1e-19ffbb7516af admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.241606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae8d86-caf5-44be-8d1e-19ffbb7516af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:52:04.241925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae8d86-caf5-44be-8d1e-19ffbb7516af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:04.243550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae8d86-caf5-44be-8d1e-19ffbb7516af admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:04.257897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1166/2348] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:52:04.262146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5b202-ac3a-4599-abbc-470085b5ea12 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:04.272696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5b202-ac3a-4599-abbc-470085b5ea12 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:04.277504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50cd45-c6e5-4e71-9554-c8c46435fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:04.278096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50cd45-c6e5-4e71-9554-c8c46435fb77 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.278371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50cd45-c6e5-4e71-9554-c8c46435fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.278642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50cd45-c6e5-4e71-9554-c8c46435fb77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.279332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50cd45-c6e5-4e71-9554-c8c46435fb77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:04.281291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50cd45-c6e5-4e71-9554-c8c46435fb77 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:04.314393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1183/2349] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:52:04 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:52:04.315605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae8d86-caf5-44be-8d1e-19ffbb7516af admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:04.327082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae8d86-caf5-44be-8d1e-19ffbb7516af admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:04.355896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:04.389982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fda415-7e09-4307-9c4a-4c79df6829f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:04.390593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fda415-7e09-4307-9c4a-4c79df6829f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.390880 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fda415-7e09-4307-9c4a-4c79df6829f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.391156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fda415-7e09-4307-9c4a-4c79df6829f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.391812 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fda415-7e09-4307-9c4a-4c79df6829f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:04.393809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fda415-7e09-4307-9c4a-4c79df6829f3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:04.415705 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1167/2350] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:52:04 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:52:04.421281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50cd45-c6e5-4e71-9554-c8c46435fb77 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:04.432078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50cd45-c6e5-4e71-9554-c8c46435fb77 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:04.438254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672037f-9dcc-4f99-996c-71e6f7e45ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:04.438896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672037f-9dcc-4f99-996c-71e6f7e45ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.439183 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672037f-9dcc-4f99-996c-71e6f7e45ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.439463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672037f-9dcc-4f99-996c-71e6f7e45ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.440135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672037f-9dcc-4f99-996c-71e6f7e45ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:04.441815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672037f-9dcc-4f99-996c-71e6f7e45ec8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:04.453714 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1184/2351] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:52:04.464584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fda415-7e09-4307-9c4a-4c79df6829f3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:04.478852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fda415-7e09-4307-9c4a-4c79df6829f3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:04.484350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8288a2-6cc0-4f6a-a95d-53f18ceb5ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:04.485003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8288a2-6cc0-4f6a-a95d-53f18ceb5ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.485292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8288a2-6cc0-4f6a-a95d-53f18ceb5ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.485578 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8288a2-6cc0-4f6a-a95d-53f18ceb5ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.486224 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8288a2-6cc0-4f6a-a95d-53f18ceb5ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:04.487996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8288a2-6cc0-4f6a-a95d-53f18ceb5ff7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:04.504966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1168/2352] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:52:04.509399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672037f-9dcc-4f99-996c-71e6f7e45ec8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:04.519962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672037f-9dcc-4f99-996c-71e6f7e45ec8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:04.524813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f787a-6e70-4c30-b33c-9115cde921bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:04.525489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f787a-6e70-4c30-b33c-9115cde921bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.525767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f787a-6e70-4c30-b33c-9115cde921bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.526068 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f787a-6e70-4c30-b33c-9115cde921bb admin admin] PATH_INFO: `/v3/projects/f92e0a403d5a440eb82ac8fab706c4d4/users/8042093e3c0b49969ea0423eae1dd563/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.545880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f787a-6e70-4c30-b33c-9115cde921bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92e0a403d5a440eb82ac8fab706c4d4, user_id=8042093e3c0b49969ea0423eae1dd563, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:04.548609 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f787a-6e70-4c30-b33c-9115cde921bb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:04.562445 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1185/2353] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:52:04 2022] PUT /identity/v3/projects/f92e0a403d5a440eb82ac8fab706c4d4/users/8042093e3c0b49969ea0423eae1dd563/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:04.573223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22826e58-29e5-4c8a-acba-09ca6a13e081 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.573771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22826e58-29e5-4c8a-acba-09ca6a13e081 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.573771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22826e58-29e5-4c8a-acba-09ca6a13e081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.602939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-22826e58-29e5-4c8a-acba-09ca6a13e081 None None] MFA Rules not processed for user `8042093e3c0b49969ea0423eae1dd563`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:04.604205 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-22826e58-29e5-4c8a-acba-09ca6a13e081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:04.675732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1169/2354] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:04.680383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f787a-6e70-4c30-b33c-9115cde921bb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:04.749938 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f787a-6e70-4c30-b33c-9115cde921bb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:04.754988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbee1f-f782-45e1-b7e5-4abe8b501e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:04.755623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbee1f-f782-45e1-b7e5-4abe8b501e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.755892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbee1f-f782-45e1-b7e5-4abe8b501e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.756197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbee1f-f782-45e1-b7e5-4abe8b501e34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.756885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbee1f-f782-45e1-b7e5-4abe8b501e34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:04.758817 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbee1f-f782-45e1-b7e5-4abe8b501e34 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:04.786253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1186/2355] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:52:04 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:52:04.790298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22826e58-29e5-4c8a-acba-09ca6a13e081 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:04.802610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22826e58-29e5-4c8a-acba-09ca6a13e081 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:04.865092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422bad3-e27b-40c0-9ca9-64cc59be3352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:04.865092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422bad3-e27b-40c0-9ca9-64cc59be3352 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.865092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422bad3-e27b-40c0-9ca9-64cc59be3352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.865092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422bad3-e27b-40c0-9ca9-64cc59be3352 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.865643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422bad3-e27b-40c0-9ca9-64cc59be3352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:04.865643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422bad3-e27b-40c0-9ca9-64cc59be3352 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:04.894876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1170/2356] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:52:04 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:52:04.896213 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbee1f-f782-45e1-b7e5-4abe8b501e34 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:04.909087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbee1f-f782-45e1-b7e5-4abe8b501e34 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:04.914410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40940d-d847-408f-bbeb-323e86360ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:04.915082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40940d-d847-408f-bbeb-323e86360ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.915366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40940d-d847-408f-bbeb-323e86360ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.915669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40940d-d847-408f-bbeb-323e86360ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.916358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40940d-d847-408f-bbeb-323e86360ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:04.918215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40940d-d847-408f-bbeb-323e86360ec8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:04.930605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1187/2357] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:52:04.935174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422bad3-e27b-40c0-9ca9-64cc59be3352 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:04.946807 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422bad3-e27b-40c0-9ca9-64cc59be3352 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:04.953574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be43f5-ce3f-4575-9e6d-a185bfd9a33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:04.954547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be43f5-ce3f-4575-9e6d-a185bfd9a33e admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:04.954838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be43f5-ce3f-4575-9e6d-a185bfd9a33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:04.955118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be43f5-ce3f-4575-9e6d-a185bfd9a33e admin admin] PATH_INFO: `/v3/projects/f8375d07828b4c41af7e6b26b5fb7c71/users/b30d1700938c47acbbc4ffc51f20808a/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:04.972057 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be43f5-ce3f-4575-9e6d-a185bfd9a33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8375d07828b4c41af7e6b26b5fb7c71, user_id=b30d1700938c47acbbc4ffc51f20808a, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:04.974936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be43f5-ce3f-4575-9e6d-a185bfd9a33e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:04.990579 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1171/2358] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:52:04 2022] PUT /identity/v3/projects/f8375d07828b4c41af7e6b26b5fb7c71/users/b30d1700938c47acbbc4ffc51f20808a/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:04.994562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40940d-d847-408f-bbeb-323e86360ec8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:05.006773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40940d-d847-408f-bbeb-323e86360ec8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:05.028750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:05.073528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a1086-a939-4f20-9fd2-1985869ff60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:05.074145 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a1086-a939-4f20-9fd2-1985869ff60f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:05.074424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a1086-a939-4f20-9fd2-1985869ff60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:05.074697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a1086-a939-4f20-9fd2-1985869ff60f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:05.075341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a1086-a939-4f20-9fd2-1985869ff60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:05.077588 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a1086-a939-4f20-9fd2-1985869ff60f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:05.088807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1188/2359] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:52:05.093894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be43f5-ce3f-4575-9e6d-a185bfd9a33e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:05.099632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:05.106702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be43f5-ce3f-4575-9e6d-a185bfd9a33e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:05.112675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae7e28-6864-419a-bb21-6c95bcd40ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:05.113358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae7e28-6864-419a-bb21-6c95bcd40ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:05.113646 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae7e28-6864-419a-bb21-6c95bcd40ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:05.113928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae7e28-6864-419a-bb21-6c95bcd40ebc admin admin] PATH_INFO: `/v3/projects/f8375d07828b4c41af7e6b26b5fb7c71/users/b30d1700938c47acbbc4ffc51f20808a/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:05.115906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae7e28-6864-419a-bb21-6c95bcd40ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8375d07828b4c41af7e6b26b5fb7c71, user_id=b30d1700938c47acbbc4ffc51f20808a, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:05.118895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae7e28-6864-419a-bb21-6c95bcd40ebc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:05.134493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1172/2360] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:52:05 2022] PUT /identity/v3/projects/f8375d07828b4c41af7e6b26b5fb7c71/users/b30d1700938c47acbbc4ffc51f20808a/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:05.147697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f607f6-bfaf-4c9e-b4e8-9499d8132791 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:05.147910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f607f6-bfaf-4c9e-b4e8-9499d8132791 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:05.148201 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f607f6-bfaf-4c9e-b4e8-9499d8132791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:05.186455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-90f607f6-bfaf-4c9e-b4e8-9499d8132791 None None] MFA Rules not processed for user `b30d1700938c47acbbc4ffc51f20808a`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:05.187953 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-90f607f6-bfaf-4c9e-b4e8-9499d8132791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:05.260117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1189/2361] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:05.272383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae7e28-6864-419a-bb21-6c95bcd40ebc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:05.274269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-10ae7e28-6864-419a-bb21-6c95bcd40ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:05.335128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae7e28-6864-419a-bb21-6c95bcd40ebc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:05.338980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e661b-5993-4972-bc83-2220d15496ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:05.339618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e661b-5993-4972-bc83-2220d15496ee service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:05.339898 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e661b-5993-4972-bc83-2220d15496ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:05.340170 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e661b-5993-4972-bc83-2220d15496ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:05.351150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e661b-5993-4972-bc83-2220d15496ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:05.352670 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e661b-5993-4972-bc83-2220d15496ee service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:05.364096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1173/2362] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:52:05.453446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f607f6-bfaf-4c9e-b4e8-9499d8132791 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:05.463814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f607f6-bfaf-4c9e-b4e8-9499d8132791 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:05.468051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766039cf-6938-452c-8130-d70ff78e52fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:05.468765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766039cf-6938-452c-8130-d70ff78e52fe service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:05.469054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766039cf-6938-452c-8130-d70ff78e52fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:05.469331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766039cf-6938-452c-8130-d70ff78e52fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:05.532953 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766039cf-6938-452c-8130-d70ff78e52fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:05.534314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766039cf-6938-452c-8130-d70ff78e52fe service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:05.544338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1190/2363] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:52:10.452389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e661b-5993-4972-bc83-2220d15496ee service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:10.516322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e661b-5993-4972-bc83-2220d15496ee service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:10.521070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:10.521702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:10.521981 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:10.522254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] PATH_INFO: `/v3/users/8042093e3c0b49969ea0423eae1dd563` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:10.523394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8042093e3c0b49969ea0423eae1dd563)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:10.525344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:10.572483 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8042093e3c0b49969ea0423eae1dd563'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:10.573183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8042093e3c0b49969ea0423eae1dd563'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:10.580593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8042093e3c0b49969ea0423eae1dd563'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:10.588015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8042093e3c0b49969ea0423eae1dd563'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:10.588686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8042093e3c0b49969ea0423eae1dd563'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:10.606628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1174/2364] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:52:10 2022] DELETE /identity/v3/users/8042093e3c0b49969ea0423eae1dd563 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:10.611667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cff20-92b9-4542-881c-e4288d42f1a7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:10.612160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cff20-92b9-4542-881c-e4288d42f1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:10.612257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cff20-92b9-4542-881c-e4288d42f1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:10.631455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c34cff20-92b9-4542-881c-e4288d42f1a7 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:10.632708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c34cff20-92b9-4542-881c-e4288d42f1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:10.674209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:10.694514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1191/2365] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:10.704335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:10.705836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:10.720010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:10.765968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040f9c9-9a86-4b6f-bd9b-ebcf317ed09f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:10.769756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e85110-e939-4d76-a2b0-29ba8150c562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:10.770453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85110-e939-4d76-a2b0-29ba8150c562 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:10.770739 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85110-e939-4d76-a2b0-29ba8150c562 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:10.771014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85110-e939-4d76-a2b0-29ba8150c562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:10.782529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e85110-e939-4d76-a2b0-29ba8150c562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:10.783895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e85110-e939-4d76-a2b0-29ba8150c562 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:10.794160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1175/2366] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:52:11.118440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cff20-92b9-4542-881c-e4288d42f1a7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:11.120168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c34cff20-92b9-4542-881c-e4288d42f1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:11.141920 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cff20-92b9-4542-881c-e4288d42f1a7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:11.148684 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:11.149410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:11.149760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:11.150105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] PATH_INFO: `/v3/projects/f92e0a403d5a440eb82ac8fab706c4d4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:11.151514 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92e0a403d5a440eb82ac8fab706c4d4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:11.153713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:11.250268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92e0a403d5a440eb82ac8fab706c4d4', 'request_id': 'req-e9315a4f-c061-4a1f-bb68-199565baa95d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.258998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92e0a403d5a440eb82ac8fab706c4d4', 'request_id': 'req-e9315a4f-c061-4a1f-bb68-199565baa95d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.267169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92e0a403d5a440eb82ac8fab706c4d4', 'request_id': 'req-e9315a4f-c061-4a1f-bb68-199565baa95d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.291242 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] The token cache is being invalidate because project f92e0a403d5a440eb82ac8fab706c4d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:52:11.291540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.293499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.296408 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1192/2367] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:52:11 2022] DELETE /identity/v3/projects/f92e0a403d5a440eb82ac8fab706c4d4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:11.300772 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e85110-e939-4d76-a2b0-29ba8150c562 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:11.312610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e85110-e939-4d76-a2b0-29ba8150c562 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:11.375445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af72a69-858f-4ab1-932f-208f48cddb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:11.376202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af72a69-858f-4ab1-932f-208f48cddb69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:11.376586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af72a69-858f-4ab1-932f-208f48cddb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:11.376961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af72a69-858f-4ab1-932f-208f48cddb69 admin admin] PATH_INFO: `/v3/users/b30d1700938c47acbbc4ffc51f20808a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:11.378780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af72a69-858f-4ab1-932f-208f48cddb69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30d1700938c47acbbc4ffc51f20808a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:11.381041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af72a69-858f-4ab1-932f-208f48cddb69 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:11.431355 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8af72a69-858f-4ab1-932f-208f48cddb69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30d1700938c47acbbc4ffc51f20808a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.432014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8af72a69-858f-4ab1-932f-208f48cddb69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30d1700938c47acbbc4ffc51f20808a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.439089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8af72a69-858f-4ab1-932f-208f48cddb69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30d1700938c47acbbc4ffc51f20808a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.447491 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8af72a69-858f-4ab1-932f-208f48cddb69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30d1700938c47acbbc4ffc51f20808a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.448144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8af72a69-858f-4ab1-932f-208f48cddb69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30d1700938c47acbbc4ffc51f20808a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.465552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1176/2368] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:52:11 2022] DELETE /identity/v3/users/b30d1700938c47acbbc4ffc51f20808a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:11.752984 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:11.754501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:11.768919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9315a4f-c061-4a1f-bb68-199565baa95d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:11.828371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:11.829022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:11.829471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:11.829591 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] PATH_INFO: `/v3/projects/f8375d07828b4c41af7e6b26b5fb7c71` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:11.830666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8375d07828b4c41af7e6b26b5fb7c71)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:11.832676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:11.879898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8375d07828b4c41af7e6b26b5fb7c71', 'request_id': 'req-13bc9d44-4e18-45e1-b5c9-d6b183899255'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.886155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8375d07828b4c41af7e6b26b5fb7c71', 'request_id': 'req-13bc9d44-4e18-45e1-b5c9-d6b183899255'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.893230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8375d07828b4c41af7e6b26b5fb7c71', 'request_id': 'req-13bc9d44-4e18-45e1-b5c9-d6b183899255'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.901430 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:11.906458 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] The token cache is being invalidate because project f8375d07828b4c41af7e6b26b5fb7c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:52:11.906975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.907757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-13bc9d44-4e18-45e1-b5c9-d6b183899255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:11.909709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1193/2369] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:52:11 2022] DELETE /identity/v3/projects/f8375d07828b4c41af7e6b26b5fb7c71 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:11.915103 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6eb2b8-875d-4b0a-bc7f-331155baa3fc None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:11.915408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6eb2b8-875d-4b0a-bc7f-331155baa3fc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:11.915661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6eb2b8-875d-4b0a-bc7f-331155baa3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:11.939251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1b6eb2b8-875d-4b0a-bc7f-331155baa3fc None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:11.940678 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1b6eb2b8-875d-4b0a-bc7f-331155baa3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:11.972007 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:12.006281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1177/2370] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:12.014351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94754e90-28eb-44f2-b157-312d11756b68 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.014641 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94754e90-28eb-44f2-b157-312d11756b68 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.014983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94754e90-28eb-44f2-b157-312d11756b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.036440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-94754e90-28eb-44f2-b157-312d11756b68 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:12.037766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-94754e90-28eb-44f2-b157-312d11756b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:12.046776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1194/2371] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:12.051106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6eb2b8-875d-4b0a-bc7f-331155baa3fc None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:12.061225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6eb2b8-875d-4b0a-bc7f-331155baa3fc None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:12.065854 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad519c96-3742-4856-885f-02a98df69caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:12.066457 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad519c96-3742-4856-885f-02a98df69caa admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.066745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad519c96-3742-4856-885f-02a98df69caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.067024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad519c96-3742-4856-885f-02a98df69caa admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.067709 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad519c96-3742-4856-885f-02a98df69caa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:52:12.068021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad519c96-3742-4856-885f-02a98df69caa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:12.069939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad519c96-3742-4856-885f-02a98df69caa admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:12.081728 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1178/2372] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:52:12.086046 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94754e90-28eb-44f2-b157-312d11756b68 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:12.096404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94754e90-28eb-44f2-b157-312d11756b68 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:12.101528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b06f3-3a5b-4936-aed0-e7dd63b09ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:12.102208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b06f3-3a5b-4936-aed0-e7dd63b09ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.102503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b06f3-3a5b-4936-aed0-e7dd63b09ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.102777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b06f3-3a5b-4936-aed0-e7dd63b09ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.103407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b06f3-3a5b-4936-aed0-e7dd63b09ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:12.105361 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b06f3-3a5b-4936-aed0-e7dd63b09ae4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:12.136129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1195/2373] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:52:12 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:52:12.140864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad519c96-3742-4856-885f-02a98df69caa admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:12.151308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad519c96-3742-4856-885f-02a98df69caa admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:12.219974 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9430bfb-7390-42ff-8d96-0c0bbd12f37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:12.220616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9430bfb-7390-42ff-8d96-0c0bbd12f37c admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.220904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9430bfb-7390-42ff-8d96-0c0bbd12f37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.221187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9430bfb-7390-42ff-8d96-0c0bbd12f37c admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.221830 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9430bfb-7390-42ff-8d96-0c0bbd12f37c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:12.223788 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9430bfb-7390-42ff-8d96-0c0bbd12f37c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:12.247716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1179/2374] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:52:12 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:52:12.252355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b06f3-3a5b-4936-aed0-e7dd63b09ae4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:12.263856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b06f3-3a5b-4936-aed0-e7dd63b09ae4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:12.271715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106d4d4-4292-4b9b-a085-f95ffb88cb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:12.272468 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106d4d4-4292-4b9b-a085-f95ffb88cb6a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.272890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106d4d4-4292-4b9b-a085-f95ffb88cb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.273152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106d4d4-4292-4b9b-a085-f95ffb88cb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.274011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106d4d4-4292-4b9b-a085-f95ffb88cb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:12.275602 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106d4d4-4292-4b9b-a085-f95ffb88cb6a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:12.287992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1196/2375] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:52:12.292057 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9430bfb-7390-42ff-8d96-0c0bbd12f37c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:12.303966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9430bfb-7390-42ff-8d96-0c0bbd12f37c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:12.308663 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab2828-faef-4b95-a198-342ea34c66af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:12.309301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab2828-faef-4b95-a198-342ea34c66af admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.309586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab2828-faef-4b95-a198-342ea34c66af admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.309861 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab2828-faef-4b95-a198-342ea34c66af admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.310622 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab2828-faef-4b95-a198-342ea34c66af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:12.312218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab2828-faef-4b95-a198-342ea34c66af admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:12.323843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1180/2376] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:52:12.328165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106d4d4-4292-4b9b-a085-f95ffb88cb6a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:12.339047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106d4d4-4292-4b9b-a085-f95ffb88cb6a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:12.345610 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f57b38-f8cf-4a57-b28b-1e38c1c2c622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:12.346352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f57b38-f8cf-4a57-b28b-1e38c1c2c622 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.346704 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f57b38-f8cf-4a57-b28b-1e38c1c2c622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.346992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f57b38-f8cf-4a57-b28b-1e38c1c2c622 admin admin] PATH_INFO: `/v3/projects/880c0db780134709b33a1c6de16dfae2/users/bd7d7fdf9dea486d9a6de29ca7e59fef/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.364115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f57b38-f8cf-4a57-b28b-1e38c1c2c622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=880c0db780134709b33a1c6de16dfae2, user_id=bd7d7fdf9dea486d9a6de29ca7e59fef, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:12.366862 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f57b38-f8cf-4a57-b28b-1e38c1c2c622 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:12.381364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1197/2377] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:52:12 2022] PUT /identity/v3/projects/880c0db780134709b33a1c6de16dfae2/users/bd7d7fdf9dea486d9a6de29ca7e59fef/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:12.393242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34e6e-0c24-4fbd-992f-659a6ce048d2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.393479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34e6e-0c24-4fbd-992f-659a6ce048d2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.393762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34e6e-0c24-4fbd-992f-659a6ce048d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.425597 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cff34e6e-0c24-4fbd-992f-659a6ce048d2 None None] MFA Rules not processed for user `bd7d7fdf9dea486d9a6de29ca7e59fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:12.426846 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cff34e6e-0c24-4fbd-992f-659a6ce048d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:12.514325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1181/2378] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:12.518927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f57b38-f8cf-4a57-b28b-1e38c1c2c622 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:12.625953 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f57b38-f8cf-4a57-b28b-1e38c1c2c622 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:12.640081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b2bd-bcd1-4fd6-9f7a-2a8d39a905ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:12.640081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b2bd-bcd1-4fd6-9f7a-2a8d39a905ff admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.640081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b2bd-bcd1-4fd6-9f7a-2a8d39a905ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.640081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b2bd-bcd1-4fd6-9f7a-2a8d39a905ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.641396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877b2bd-bcd1-4fd6-9f7a-2a8d39a905ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:12.643897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877b2bd-bcd1-4fd6-9f7a-2a8d39a905ff admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:12.679158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1198/2379] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:52:12 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:52:12.689062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff34e6e-0c24-4fbd-992f-659a6ce048d2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:12.701655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff34e6e-0c24-4fbd-992f-659a6ce048d2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:12.737924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:12.774062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723802c4-c0a7-4489-94d2-64bcf6f18e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:12.774714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723802c4-c0a7-4489-94d2-64bcf6f18e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.775047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723802c4-c0a7-4489-94d2-64bcf6f18e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.775333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723802c4-c0a7-4489-94d2-64bcf6f18e16 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.776008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723802c4-c0a7-4489-94d2-64bcf6f18e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:12.778087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723802c4-c0a7-4489-94d2-64bcf6f18e16 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:12.804378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1182/2380] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:52:12 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:52:12.808870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b2bd-bcd1-4fd6-9f7a-2a8d39a905ff admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:12.821070 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b2bd-bcd1-4fd6-9f7a-2a8d39a905ff admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:12.826244 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73ed5e-7bb1-4a6b-9f46-35d3a53d317b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:12.826889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73ed5e-7bb1-4a6b-9f46-35d3a53d317b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.827171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73ed5e-7bb1-4a6b-9f46-35d3a53d317b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.827442 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73ed5e-7bb1-4a6b-9f46-35d3a53d317b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.828136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73ed5e-7bb1-4a6b-9f46-35d3a53d317b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:12.829855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73ed5e-7bb1-4a6b-9f46-35d3a53d317b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:12.842107 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1199/2381] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:52:12.847090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723802c4-c0a7-4489-94d2-64bcf6f18e16 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:12.859221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723802c4-c0a7-4489-94d2-64bcf6f18e16 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:12.863961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b7361-3882-4de8-b518-e329c21a7581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:12.864671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b7361-3882-4de8-b518-e329c21a7581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.864941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b7361-3882-4de8-b518-e329c21a7581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.865216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b7361-3882-4de8-b518-e329c21a7581 admin admin] PATH_INFO: `/v3/projects/289a92a7c1cb4e5d9d21825ec466714e/users/f7bcdb0440024b90abe3dd2d43c982e0/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.881516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b7361-3882-4de8-b518-e329c21a7581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289a92a7c1cb4e5d9d21825ec466714e, user_id=f7bcdb0440024b90abe3dd2d43c982e0, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:12.884219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b7361-3882-4de8-b518-e329c21a7581 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:12.904991 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1183/2382] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:52:12 2022] PUT /identity/v3/projects/289a92a7c1cb4e5d9d21825ec466714e/users/f7bcdb0440024b90abe3dd2d43c982e0/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:12.909372 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73ed5e-7bb1-4a6b-9f46-35d3a53d317b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:12.923380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73ed5e-7bb1-4a6b-9f46-35d3a53d317b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:12.989382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f584c3-c54f-48d3-9aad-e8ab09b69275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:12.990040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f584c3-c54f-48d3-9aad-e8ab09b69275 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:12.990336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f584c3-c54f-48d3-9aad-e8ab09b69275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:12.990618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f584c3-c54f-48d3-9aad-e8ab09b69275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:12.991302 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f584c3-c54f-48d3-9aad-e8ab09b69275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:12.993100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f584c3-c54f-48d3-9aad-e8ab09b69275 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:13.004974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1200/2383] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:52:13.009405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b7361-3882-4de8-b518-e329c21a7581 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:13.020924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b7361-3882-4de8-b518-e329c21a7581 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:13.026519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3574fac-da30-459f-992d-22365f84e4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:13.027414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3574fac-da30-459f-992d-22365f84e4d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:13.027771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3574fac-da30-459f-992d-22365f84e4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:13.028084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3574fac-da30-459f-992d-22365f84e4d7 admin admin] PATH_INFO: `/v3/projects/289a92a7c1cb4e5d9d21825ec466714e/users/f7bcdb0440024b90abe3dd2d43c982e0/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:13.031012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3574fac-da30-459f-992d-22365f84e4d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289a92a7c1cb4e5d9d21825ec466714e, user_id=f7bcdb0440024b90abe3dd2d43c982e0, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:13.034053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3574fac-da30-459f-992d-22365f84e4d7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:13.051877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1184/2384] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:52:13 2022] PUT /identity/v3/projects/289a92a7c1cb4e5d9d21825ec466714e/users/f7bcdb0440024b90abe3dd2d43c982e0/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:13.064962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc604f-68f3-412a-a226-c262fa06b8f1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:13.065341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc604f-68f3-412a-a226-c262fa06b8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:13.065750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc604f-68f3-412a-a226-c262fa06b8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:13.075573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:13.099598 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-95fc604f-68f3-412a-a226-c262fa06b8f1 None None] MFA Rules not processed for user `f7bcdb0440024b90abe3dd2d43c982e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:13.100872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-95fc604f-68f3-412a-a226-c262fa06b8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:13.186419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1201/2385] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:13.198147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3574fac-da30-459f-992d-22365f84e4d7 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:13.200046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d3574fac-da30-459f-992d-22365f84e4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:13.269413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3574fac-da30-459f-992d-22365f84e4d7 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:13.274900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743485a7-d2ed-4346-b419-212e0bc7e773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:13.275605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743485a7-d2ed-4346-b419-212e0bc7e773 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:13.275938 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743485a7-d2ed-4346-b419-212e0bc7e773 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:13.276300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743485a7-d2ed-4346-b419-212e0bc7e773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:13.343472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743485a7-d2ed-4346-b419-212e0bc7e773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:13.345009 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743485a7-d2ed-4346-b419-212e0bc7e773 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:13.356398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1185/2386] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:52:17.791121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc604f-68f3-412a-a226-c262fa06b8f1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:17.802196 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc604f-68f3-412a-a226-c262fa06b8f1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:17.806114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50b88d-f8e9-4a9f-a636-c3e6bd881b75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:17.806719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b88d-f8e9-4a9f-a636-c3e6bd881b75 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:17.806999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b88d-f8e9-4a9f-a636-c3e6bd881b75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:17.807266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b88d-f8e9-4a9f-a636-c3e6bd881b75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:17.817459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50b88d-f8e9-4a9f-a636-c3e6bd881b75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:17.819158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50b88d-f8e9-4a9f-a636-c3e6bd881b75 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:17.833078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1202/2387] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:52:22.883339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743485a7-d2ed-4346-b419-212e0bc7e773 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:22.925873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:22.953714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743485a7-d2ed-4346-b419-212e0bc7e773 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:22.958687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:22.959665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:22.960051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:22.960519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] PATH_INFO: `/v3/users/bd7d7fdf9dea486d9a6de29ca7e59fef` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:22.961781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7d7fdf9dea486d9a6de29ca7e59fef)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:22.963788 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:23.016314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7d7fdf9dea486d9a6de29ca7e59fef'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.017048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7d7fdf9dea486d9a6de29ca7e59fef'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.026379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7d7fdf9dea486d9a6de29ca7e59fef'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.038157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7d7fdf9dea486d9a6de29ca7e59fef'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.038958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7d7fdf9dea486d9a6de29ca7e59fef'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.057244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1186/2388] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:52:22 2022] DELETE /identity/v3/users/bd7d7fdf9dea486d9a6de29ca7e59fef => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:52:23.062724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0e7f0-e224-4ce8-8928-1d23abb0f1f1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:23.063268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0e7f0-e224-4ce8-8928-1d23abb0f1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:23.063268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0e7f0-e224-4ce8-8928-1d23abb0f1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:23.084523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-91a0e7f0-e224-4ce8-8928-1d23abb0f1f1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:23.085773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-91a0e7f0-e224-4ce8-8928-1d23abb0f1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:23.149530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1203/2389] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:52:23.161054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:23.162659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:23.243505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49c114-6fa2-4330-a402-a64d6444e9ca admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:23.247063 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2f32f-cc36-4c4f-8c5d-fba1a815a149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:23.249180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2f32f-cc36-4c4f-8c5d-fba1a815a149 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:23.249180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2f32f-cc36-4c4f-8c5d-fba1a815a149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:23.249460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2f32f-cc36-4c4f-8c5d-fba1a815a149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:23.263284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2f32f-cc36-4c4f-8c5d-fba1a815a149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:23.266142 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2f32f-cc36-4c4f-8c5d-fba1a815a149 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:23.279715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1187/2390] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:52:23.707485 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0e7f0-e224-4ce8-8928-1d23abb0f1f1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:23.710631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-91a0e7f0-e224-4ce8-8928-1d23abb0f1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:23.723853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0e7f0-e224-4ce8-8928-1d23abb0f1f1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:23.728623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:23.729267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:23.730394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:23.730394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] PATH_INFO: `/v3/projects/880c0db780134709b33a1c6de16dfae2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:23.730940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=880c0db780134709b33a1c6de16dfae2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:23.733718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:23.791083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880c0db780134709b33a1c6de16dfae2', 'request_id': 'req-609a2e03-c571-4e5e-a141-773bb466f61e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.802519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880c0db780134709b33a1c6de16dfae2', 'request_id': 'req-609a2e03-c571-4e5e-a141-773bb466f61e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.810506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880c0db780134709b33a1c6de16dfae2', 'request_id': 'req-609a2e03-c571-4e5e-a141-773bb466f61e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.823282 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:23.829923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] The token cache is being invalidate because project 880c0db780134709b33a1c6de16dfae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:52:23.830666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.832206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.837246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1204/2391] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:52:23 2022] DELETE /identity/v3/projects/880c0db780134709b33a1c6de16dfae2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:23.839451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2f32f-cc36-4c4f-8c5d-fba1a815a149 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:23.850424 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2f32f-cc36-4c4f-8c5d-fba1a815a149 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:23.909895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:23.917484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494fae6-0d4e-4bde-b91d-b4bef59250e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:23.918262 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494fae6-0d4e-4bde-b91d-b4bef59250e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:23.918613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494fae6-0d4e-4bde-b91d-b4bef59250e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:23.918898 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494fae6-0d4e-4bde-b91d-b4bef59250e3 admin admin] PATH_INFO: `/v3/users/f7bcdb0440024b90abe3dd2d43c982e0` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:23.920992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494fae6-0d4e-4bde-b91d-b4bef59250e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7bcdb0440024b90abe3dd2d43c982e0)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:23.923398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494fae6-0d4e-4bde-b91d-b4bef59250e3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:23.988555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b494fae6-0d4e-4bde-b91d-b4bef59250e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7bcdb0440024b90abe3dd2d43c982e0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.989575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b494fae6-0d4e-4bde-b91d-b4bef59250e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7bcdb0440024b90abe3dd2d43c982e0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:23.999595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b494fae6-0d4e-4bde-b91d-b4bef59250e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7bcdb0440024b90abe3dd2d43c982e0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:24.008615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b494fae6-0d4e-4bde-b91d-b4bef59250e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7bcdb0440024b90abe3dd2d43c982e0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:24.009401 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b494fae6-0d4e-4bde-b91d-b4bef59250e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7bcdb0440024b90abe3dd2d43c982e0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:24.032248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1188/2392] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:52:23 2022] DELETE /identity/v3/users/f7bcdb0440024b90abe3dd2d43c982e0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:24.354028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:24.354979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:24.370666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a2e03-c571-4e5e-a141-773bb466f61e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:24.469383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:24.470037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:24.470340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:24.470603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] PATH_INFO: `/v3/projects/289a92a7c1cb4e5d9d21825ec466714e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:24.472007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] RBAC: Authorizing `identity:delete_project(project_id=289a92a7c1cb4e5d9d21825ec466714e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:24.474677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:24.549170 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '289a92a7c1cb4e5d9d21825ec466714e', 'request_id': 'req-98bca891-f24c-484b-93e7-4634caacb63e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:24.557696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '289a92a7c1cb4e5d9d21825ec466714e', 'request_id': 'req-98bca891-f24c-484b-93e7-4634caacb63e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:24.571345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '289a92a7c1cb4e5d9d21825ec466714e', 'request_id': 'req-98bca891-f24c-484b-93e7-4634caacb63e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:24.597412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] The token cache is being invalidate because project 289a92a7c1cb4e5d9d21825ec466714e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:52:24.598411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:24.599235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-98bca891-f24c-484b-93e7-4634caacb63e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:24.604529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1205/2393] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:52:24 2022] DELETE /identity/v3/projects/289a92a7c1cb4e5d9d21825ec466714e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:24.623556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10272e40-adca-4a97-92e4-755a189df0b9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:24.624380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10272e40-adca-4a97-92e4-755a189df0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:24.624975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10272e40-adca-4a97-92e4-755a189df0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:24.650784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-10272e40-adca-4a97-92e4-755a189df0b9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:24.652154 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-10272e40-adca-4a97-92e4-755a189df0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:24.730484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1189/2394] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:24.741570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6ce57-4e59-4ca4-b471-ffb83bd0a3eb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:24.741856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6ce57-4e59-4ca4-b471-ffb83bd0a3eb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:24.742131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6ce57-4e59-4ca4-b471-ffb83bd0a3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:24.767013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-07c6ce57-4e59-4ca4-b471-ffb83bd0a3eb None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:24.768232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-07c6ce57-4e59-4ca4-b471-ffb83bd0a3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:24.776534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1206/2395] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:24.781623 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10272e40-adca-4a97-92e4-755a189df0b9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:24.792581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10272e40-adca-4a97-92e4-755a189df0b9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:24.798529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cfb81-9d34-41d6-bfe4-de13e395d1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:24.799239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cfb81-9d34-41d6-bfe4-de13e395d1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:24.799456 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cfb81-9d34-41d6-bfe4-de13e395d1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:24.799731 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cfb81-9d34-41d6-bfe4-de13e395d1e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:24.800426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cfb81-9d34-41d6-bfe4-de13e395d1e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:52:24.800780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cfb81-9d34-41d6-bfe4-de13e395d1e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:24.802467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cfb81-9d34-41d6-bfe4-de13e395d1e9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:24.818241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1190/2396] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:52:24.823200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6ce57-4e59-4ca4-b471-ffb83bd0a3eb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:24.842650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6ce57-4e59-4ca4-b471-ffb83bd0a3eb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:24.855683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756656-aba9-4d3d-a782-766ca2e65bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:24.856416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756656-aba9-4d3d-a782-766ca2e65bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:24.857569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756656-aba9-4d3d-a782-766ca2e65bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:24.858020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756656-aba9-4d3d-a782-766ca2e65bf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:24.858492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d756656-aba9-4d3d-a782-766ca2e65bf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:24.860827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d756656-aba9-4d3d-a782-766ca2e65bf6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:24.897924 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1207/2397] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:52:24 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:52:24.903444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cfb81-9d34-41d6-bfe4-de13e395d1e9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:24.923783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cfb81-9d34-41d6-bfe4-de13e395d1e9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:25.057330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ae7e3-fad9-4cec-8415-9156deb1673f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:25.059514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ae7e3-fad9-4cec-8415-9156deb1673f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:25.062925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ae7e3-fad9-4cec-8415-9156deb1673f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:25.063813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ae7e3-fad9-4cec-8415-9156deb1673f admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:25.064648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ae7e3-fad9-4cec-8415-9156deb1673f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:25.067607 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ae7e3-fad9-4cec-8415-9156deb1673f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:25.088616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:25.123912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1191/2398] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:52:24 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:52:25.129292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756656-aba9-4d3d-a782-766ca2e65bf6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:25.147722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756656-aba9-4d3d-a782-766ca2e65bf6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:25.163483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcaf3f-3167-4dc4-a7ab-0d71490d1cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:25.164247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcaf3f-3167-4dc4-a7ab-0d71490d1cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:25.164671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcaf3f-3167-4dc4-a7ab-0d71490d1cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:25.165344 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcaf3f-3167-4dc4-a7ab-0d71490d1cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:25.166558 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcaf3f-3167-4dc4-a7ab-0d71490d1cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:25.170520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcaf3f-3167-4dc4-a7ab-0d71490d1cd4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:25.190222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1208/2399] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:52:25.194596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ae7e3-fad9-4cec-8415-9156deb1673f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:25.208982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ae7e3-fad9-4cec-8415-9156deb1673f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:25.213704 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c158b7-83b1-440f-82af-79d79333e0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:25.214317 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c158b7-83b1-440f-82af-79d79333e0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:25.214615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c158b7-83b1-440f-82af-79d79333e0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:25.214891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c158b7-83b1-440f-82af-79d79333e0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:25.215551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c158b7-83b1-440f-82af-79d79333e0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:25.218262 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c158b7-83b1-440f-82af-79d79333e0c0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:25.230669 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1192/2400] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:52:25.234907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcaf3f-3167-4dc4-a7ab-0d71490d1cd4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:25.246592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcaf3f-3167-4dc4-a7ab-0d71490d1cd4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:25.253255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453ad4-5e50-4ca8-a5dd-d672b487c47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:25.254049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453ad4-5e50-4ca8-a5dd-d672b487c47d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:25.254585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453ad4-5e50-4ca8-a5dd-d672b487c47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:25.254919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453ad4-5e50-4ca8-a5dd-d672b487c47d admin admin] PATH_INFO: `/v3/projects/2dbfe7ba453b472ba09b554b59d79281/users/ea03ffe5a78e4741a47aa8087db14053/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:25.284123 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09453ad4-5e50-4ca8-a5dd-d672b487c47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dbfe7ba453b472ba09b554b59d79281, user_id=ea03ffe5a78e4741a47aa8087db14053, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:25.288142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09453ad4-5e50-4ca8-a5dd-d672b487c47d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:25.312035 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1209/2401] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:52:25 2022] PUT /identity/v3/projects/2dbfe7ba453b472ba09b554b59d79281/users/ea03ffe5a78e4741a47aa8087db14053/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:25.320365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9460d69-59e8-4000-9d80-4e8846fa1af2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:25.320365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9460d69-59e8-4000-9d80-4e8846fa1af2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:25.320365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9460d69-59e8-4000-9d80-4e8846fa1af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:25.339444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e9460d69-59e8-4000-9d80-4e8846fa1af2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:25.340711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e9460d69-59e8-4000-9d80-4e8846fa1af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:25.434466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1193/2402] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:52:25.445244 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453ad4-5e50-4ca8-a5dd-d672b487c47d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:25.447254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-09453ad4-5e50-4ca8-a5dd-d672b487c47d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:25.489765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:25.545527 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453ad4-5e50-4ca8-a5dd-d672b487c47d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:25.549879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c84bd-f7bc-4bc5-b328-28a8826c614f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:25.550616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c84bd-f7bc-4bc5-b328-28a8826c614f service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:25.551003 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c84bd-f7bc-4bc5-b328-28a8826c614f service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:25.551345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c84bd-f7bc-4bc5-b328-28a8826c614f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:25.566133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c84bd-f7bc-4bc5-b328-28a8826c614f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:25.567968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c84bd-f7bc-4bc5-b328-28a8826c614f service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:25.582218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1210/2403] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:52:26.497277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d64e9-ad56-4c92-b139-87995e9d4cf1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:26.498320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d64e9-ad56-4c92-b139-87995e9d4cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:26.498320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d64e9-ad56-4c92-b139-87995e9d4cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:26.534288 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2d5d64e9-ad56-4c92-b139-87995e9d4cf1 None None] MFA Rules not processed for user `ea03ffe5a78e4741a47aa8087db14053`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:26.535688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2d5d64e9-ad56-4c92-b139-87995e9d4cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:26.634845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1194/2404] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:52:26.649151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c84bd-f7bc-4bc5-b328-28a8826c614f service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:26.650743 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7a9c84bd-f7bc-4bc5-b328-28a8826c614f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:26.723017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c84bd-f7bc-4bc5-b328-28a8826c614f service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:26.726792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d399c-33d8-4e38-8e8e-12c8c6420555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:26.727407 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d399c-33d8-4e38-8e8e-12c8c6420555 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:26.727691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d399c-33d8-4e38-8e8e-12c8c6420555 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:26.727976 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d399c-33d8-4e38-8e8e-12c8c6420555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:26.739414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d399c-33d8-4e38-8e8e-12c8c6420555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:26.740353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d399c-33d8-4e38-8e8e-12c8c6420555 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:26.751182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1211/2405] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:52:28.290451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd96a8-224d-4e02-90c2-f928f6791282 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:28.291045 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd96a8-224d-4e02-90c2-f928f6791282 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:28.291045 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd96a8-224d-4e02-90c2-f928f6791282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:28.319511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d9dd96a8-224d-4e02-90c2-f928f6791282 None None] MFA Rules not processed for user `ea03ffe5a78e4741a47aa8087db14053`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:28.321780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d9dd96a8-224d-4e02-90c2-f928f6791282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:28.334913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1195/2406] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:52:29.594105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d990b6-91b1-45d0-9b61-8250f3e8d6b4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:29.594989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d990b6-91b1-45d0-9b61-8250f3e8d6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:29.595334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d990b6-91b1-45d0-9b61-8250f3e8d6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:29.621995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c4d990b6-91b1-45d0-9b61-8250f3e8d6b4 None None] MFA Rules not processed for user `5de832888ffd4a69ae88cac6b632304e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:29.624466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c4d990b6-91b1-45d0-9b61-8250f3e8d6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:29.736512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1212/2407] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:36.674252 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343eb3b-cb50-4c48-bc07-ed9405793060 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:36.674918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343eb3b-cb50-4c48-bc07-ed9405793060 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:36.674918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343eb3b-cb50-4c48-bc07-ed9405793060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:36.698752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b343eb3b-cb50-4c48-bc07-ed9405793060 None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:36.699478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b343eb3b-cb50-4c48-bc07-ed9405793060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:36.807430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1196/2408] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:52:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:36.867765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d990b6-91b1-45d0-9b61-8250f3e8d6b4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:36.869441 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c4d990b6-91b1-45d0-9b61-8250f3e8d6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:36.957371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d990b6-91b1-45d0-9b61-8250f3e8d6b4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:36.963524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabd752-70cf-453c-9869-8c48612d3679 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:36.964153 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabd752-70cf-453c-9869-8c48612d3679 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:36.964423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabd752-70cf-453c-9869-8c48612d3679 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:36.964736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabd752-70cf-453c-9869-8c48612d3679 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:36.974636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabd752-70cf-453c-9869-8c48612d3679 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:36.976553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabd752-70cf-453c-9869-8c48612d3679 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:36.988897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1213/2409] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:52:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:52:46.799582 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:INSERT=1,keystone:SELECT=94 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:46.983367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:47.849862 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f003d7a-7f91-4d76-b201-80bf0ceb6e22 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:47.850515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f003d7a-7f91-4d76-b201-80bf0ceb6e22 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:47.850515 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f003d7a-7f91-4d76-b201-80bf0ceb6e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:47.876467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9f003d7a-7f91-4d76-b201-80bf0ceb6e22 None None] MFA Rules not processed for user `ea03ffe5a78e4741a47aa8087db14053`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:47.877635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9f003d7a-7f91-4d76-b201-80bf0ceb6e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:47.887015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1197/2410] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:52:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:55.925759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabd752-70cf-453c-9869-8c48612d3679 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:55.927277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bdabd752-70cf-453c-9869-8c48612d3679 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:55.939552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabd752-70cf-453c-9869-8c48612d3679 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:55.943205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825819f-dd17-469d-8cb5-3aa1aa7359b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:55.943807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825819f-dd17-469d-8cb5-3aa1aa7359b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:55.944086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825819f-dd17-469d-8cb5-3aa1aa7359b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:55.944657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825819f-dd17-469d-8cb5-3aa1aa7359b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:55.946369 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4825819f-dd17-469d-8cb5-3aa1aa7359b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:56.036650 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825819f-dd17-469d-8cb5-3aa1aa7359b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:56.038101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825819f-dd17-469d-8cb5-3aa1aa7359b6 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:56.050520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1214/2411] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:52:57.682005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd10f0-902a-4e58-833d-8b08e0b7a1fb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:57.689302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd10f0-902a-4e58-833d-8b08e0b7a1fb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:57.689734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd10f0-902a-4e58-833d-8b08e0b7a1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:57.721514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-87bd10f0-902a-4e58-833d-8b08e0b7a1fb None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:52:57.723033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-87bd10f0-902a-4e58-833d-8b08e0b7a1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:57.736689 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1198/2412] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:52:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:52:57.746222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825819f-dd17-469d-8cb5-3aa1aa7359b6 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:57.760611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825819f-dd17-469d-8cb5-3aa1aa7359b6 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:57.771020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef9e3f-9169-450e-ab6e-4d24799e9493 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:57.772245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9e3f-9169-450e-ab6e-4d24799e9493 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:57.772840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9e3f-9169-450e-ab6e-4d24799e9493 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:57.773456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9e3f-9169-450e-ab6e-4d24799e9493 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:57.801638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef9e3f-9169-450e-ab6e-4d24799e9493 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:57.803433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef9e3f-9169-450e-ab6e-4d24799e9493 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:57.817582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1215/2413] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:52:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:52:58.504013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd10f0-902a-4e58-833d-8b08e0b7a1fb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:58.518286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd10f0-902a-4e58-833d-8b08e0b7a1fb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:58.524135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88de7c6-826b-423e-902a-de13a72a982c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:58.525101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88de7c6-826b-423e-902a-de13a72a982c service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:58.525577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88de7c6-826b-423e-902a-de13a72a982c service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:58.526054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88de7c6-826b-423e-902a-de13a72a982c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:58.529096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e88de7c6-826b-423e-902a-de13a72a982c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:58.544843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88de7c6-826b-423e-902a-de13a72a982c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:58.546946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88de7c6-826b-423e-902a-de13a72a982c service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:58.561962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1199/2414] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:53:07.244503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef9e3f-9169-450e-ab6e-4d24799e9493 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:07.247294 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-42ef9e3f-9169-450e-ab6e-4d24799e9493 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:07.270071 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef9e3f-9169-450e-ab6e-4d24799e9493 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:07.276111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8e431-873e-4a4d-ae02-465d29d942f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:07.276758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8e431-873e-4a4d-ae02-465d29d942f4 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:07.277041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8e431-873e-4a4d-ae02-465d29d942f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:07.277317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8e431-873e-4a4d-ae02-465d29d942f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:07.279529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ddb8e431-873e-4a4d-ae02-465d29d942f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:07.397078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8e431-873e-4a4d-ae02-465d29d942f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:07.398743 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8e431-873e-4a4d-ae02-465d29d942f4 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:07.419919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1216/2415] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:53:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:53:08.554963 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=12 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:53:17.416895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:53:18.024462 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9080cb-368d-49d1-a754-b013273ce8ab None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:18.025002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9080cb-368d-49d1-a754-b013273ce8ab None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:18.025120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9080cb-368d-49d1-a754-b013273ce8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:18.055196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1a9080cb-368d-49d1-a754-b013273ce8ab None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:53:18.057266 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1a9080cb-368d-49d1-a754-b013273ce8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:18.070077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1200/2416] 10.222.0.60 () {58 vars in 967 bytes} [Wed Aug 31 11:53:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:53:18.085342 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8e431-873e-4a4d-ae02-465d29d942f4 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:18.113275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8e431-873e-4a4d-ae02-465d29d942f4 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:18.120544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0e5df-f019-48f6-9744-ab003ebe344e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:18.121429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0e5df-f019-48f6-9744-ab003ebe344e service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:18.121697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0e5df-f019-48f6-9744-ab003ebe344e service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:18.122031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0e5df-f019-48f6-9744-ab003ebe344e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:18.131631 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0e5df-f019-48f6-9744-ab003ebe344e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:18.133274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0e5df-f019-48f6-9744-ab003ebe344e service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:18.143003 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1217/2417] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:53:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:53:26.709050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478003c5-6104-42a1-962c-e70dfd5135ff None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:26.709573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478003c5-6104-42a1-962c-e70dfd5135ff None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:26.709672 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478003c5-6104-42a1-962c-e70dfd5135ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:26.767185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-478003c5-6104-42a1-962c-e70dfd5135ff None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:53:26.768546 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-478003c5-6104-42a1-962c-e70dfd5135ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:26.789955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1201/2418] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Aug 31 11:53:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:53:26.804664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0e5df-f019-48f6-9744-ab003ebe344e service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:26.816233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0e5df-f019-48f6-9744-ab003ebe344e service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:26.823400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa72946-9ff5-4072-a093-2d51ca5c0ce2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:26.824027 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa72946-9ff5-4072-a093-2d51ca5c0ce2 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:26.824966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa72946-9ff5-4072-a093-2d51ca5c0ce2 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:26.825764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa72946-9ff5-4072-a093-2d51ca5c0ce2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:26.834417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa72946-9ff5-4072-a093-2d51ca5c0ce2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:26.836040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa72946-9ff5-4072-a093-2d51ca5c0ce2 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:26.846239 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1218/2419] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:53:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:53:36.786747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=21 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:53:36.842611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=12 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:53:43.018540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e987b5-ff26-48b1-836a-e3c4b828c616 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:43.019093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e987b5-ff26-48b1-836a-e3c4b828c616 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:43.019189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e987b5-ff26-48b1-836a-e3c4b828c616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:43.039897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-87e987b5-ff26-48b1-836a-e3c4b828c616 None None] MFA Rules not processed for user `16216de2aa9c4a6388f94263444965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:53:43.041197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-87e987b5-ff26-48b1-836a-e3c4b828c616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:43.050706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1202/2420] 10.222.0.60 () {58 vars in 967 bytes} [Wed Aug 31 11:53:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:53:43.067858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa72946-9ff5-4072-a093-2d51ca5c0ce2 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:43.089754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa72946-9ff5-4072-a093-2d51ca5c0ce2 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:43.095439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0ffb5-ab6d-4b16-94a8-6549c4d19513 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3384c24dd081414bbf0a2e3d9f71126b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:43.096083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ffb5-ab6d-4b16-94a8-6549c4d19513 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:43.096366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ffb5-ab6d-4b16-94a8-6549c4d19513 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:43.096677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ffb5-ab6d-4b16-94a8-6549c4d19513 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:43.106459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0ffb5-ab6d-4b16-94a8-6549c4d19513 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:43.108051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0ffb5-ab6d-4b16-94a8-6549c4d19513 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:43.116685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1219/2421] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Aug 31 11:53:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:53:45.340164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e987b5-ff26-48b1-836a-e3c4b828c616 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:45.357689 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e987b5-ff26-48b1-836a-e3c4b828c616 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:45.362469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9195f7-f8e6-4c67-9b26-1135a095475a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:45.363275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9195f7-f8e6-4c67-9b26-1135a095475a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:45.363668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9195f7-f8e6-4c67-9b26-1135a095475a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:45.364028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9195f7-f8e6-4c67-9b26-1135a095475a admin admin] PATH_INFO: `/v3/users/ea03ffe5a78e4741a47aa8087db14053` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:45.365329 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9195f7-f8e6-4c67-9b26-1135a095475a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea03ffe5a78e4741a47aa8087db14053)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:45.367413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9195f7-f8e6-4c67-9b26-1135a095475a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:45.426591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ac9195f7-f8e6-4c67-9b26-1135a095475a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea03ffe5a78e4741a47aa8087db14053'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:45.427260 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ac9195f7-f8e6-4c67-9b26-1135a095475a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea03ffe5a78e4741a47aa8087db14053'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:45.436271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ac9195f7-f8e6-4c67-9b26-1135a095475a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea03ffe5a78e4741a47aa8087db14053'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:45.445192 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ac9195f7-f8e6-4c67-9b26-1135a095475a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea03ffe5a78e4741a47aa8087db14053'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:45.445461 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ac9195f7-f8e6-4c67-9b26-1135a095475a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea03ffe5a78e4741a47aa8087db14053'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:45.467668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1203/2422] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:53:45 2022] DELETE /identity/v3/users/ea03ffe5a78e4741a47aa8087db14053 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:53:45.699679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0ffb5-ab6d-4b16-94a8-6549c4d19513 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:45.718319 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0ffb5-ab6d-4b16-94a8-6549c4d19513 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:45.788031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:45.788712 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:45.788996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:45.789274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] PATH_INFO: `/v3/projects/2dbfe7ba453b472ba09b554b59d79281` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:45.790478 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dbfe7ba453b472ba09b554b59d79281)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:45.793278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:45.848556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dbfe7ba453b472ba09b554b59d79281', 'request_id': 'req-84b07b2a-742d-435d-a0b0-7893e1ae5cac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:45.856370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dbfe7ba453b472ba09b554b59d79281', 'request_id': 'req-84b07b2a-742d-435d-a0b0-7893e1ae5cac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:45.863318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dbfe7ba453b472ba09b554b59d79281', 'request_id': 'req-84b07b2a-742d-435d-a0b0-7893e1ae5cac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:45.879779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] The token cache is being invalidate because project 2dbfe7ba453b472ba09b554b59d79281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:53:45.880306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:45.881074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b07b2a-742d-435d-a0b0-7893e1ae5cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:45.882661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1220/2423] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:53:45 2022] DELETE /identity/v3/projects/2dbfe7ba453b472ba09b554b59d79281 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:53:45.890450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5ae32-35ea-418f-bac2-02d5c5e0dc81 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:45.890816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5ae32-35ea-418f-bac2-02d5c5e0dc81 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:45.891174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5ae32-35ea-418f-bac2-02d5c5e0dc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:45.911884 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e0e5ae32-35ea-418f-bac2-02d5c5e0dc81 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:53:45.913384 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e0e5ae32-35ea-418f-bac2-02d5c5e0dc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:45.981040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1204/2424] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:53:45.989021 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790f90d-7094-48ef-9ead-91a42c281b97 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:45.989306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790f90d-7094-48ef-9ead-91a42c281b97 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:45.989583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790f90d-7094-48ef-9ead-91a42c281b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:46.011258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f790f90d-7094-48ef-9ead-91a42c281b97 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:53:46.012449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f790f90d-7094-48ef-9ead-91a42c281b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:46.020624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1221/2425] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:53:46.025121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5ae32-35ea-418f-bac2-02d5c5e0dc81 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:46.034992 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5ae32-35ea-418f-bac2-02d5c5e0dc81 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:46.039570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc6323-5d45-4591-b362-2ad517875fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:46.040181 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc6323-5d45-4591-b362-2ad517875fec admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:46.040507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc6323-5d45-4591-b362-2ad517875fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:46.045958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc6323-5d45-4591-b362-2ad517875fec admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:46.049393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc6323-5d45-4591-b362-2ad517875fec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:53:46.049710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc6323-5d45-4591-b362-2ad517875fec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:46.051353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc6323-5d45-4591-b362-2ad517875fec admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:46.064971 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1205/2426] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:53:46.069800 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790f90d-7094-48ef-9ead-91a42c281b97 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:46.080528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790f90d-7094-48ef-9ead-91a42c281b97 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:46.085454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb4cb9-4de2-4edf-af02-1ce51485bb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:46.086109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb4cb9-4de2-4edf-af02-1ce51485bb50 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:46.086432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb4cb9-4de2-4edf-af02-1ce51485bb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:46.086746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb4cb9-4de2-4edf-af02-1ce51485bb50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:46.087434 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb4cb9-4de2-4edf-af02-1ce51485bb50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:46.089997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb4cb9-4de2-4edf-af02-1ce51485bb50 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:46.139945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1222/2427] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:53:46 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:53:46.141573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc6323-5d45-4591-b362-2ad517875fec admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:46.152159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc6323-5d45-4591-b362-2ad517875fec admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:46.206746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:53:46.228204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b5a4a-65c9-4b3e-bf8e-aba73efa8f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:46.229133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b5a4a-65c9-4b3e-bf8e-aba73efa8f6b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:46.229467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b5a4a-65c9-4b3e-bf8e-aba73efa8f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:46.230489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b5a4a-65c9-4b3e-bf8e-aba73efa8f6b admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:46.231473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b5a4a-65c9-4b3e-bf8e-aba73efa8f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:46.233550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b5a4a-65c9-4b3e-bf8e-aba73efa8f6b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:46.255524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1206/2428] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:53:46 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:53:46.260053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb4cb9-4de2-4edf-af02-1ce51485bb50 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:46.271106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb4cb9-4de2-4edf-af02-1ce51485bb50 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:46.276031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664703-559a-4f73-b2dd-b9d6e80a8625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:46.277471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664703-559a-4f73-b2dd-b9d6e80a8625 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:46.277803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664703-559a-4f73-b2dd-b9d6e80a8625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:46.278098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664703-559a-4f73-b2dd-b9d6e80a8625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:46.278788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15664703-559a-4f73-b2dd-b9d6e80a8625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:46.280427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15664703-559a-4f73-b2dd-b9d6e80a8625 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:46.291402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1223/2429] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:53:46.296286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b5a4a-65c9-4b3e-bf8e-aba73efa8f6b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:46.306527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b5a4a-65c9-4b3e-bf8e-aba73efa8f6b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:46.311463 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb9f4f-8d66-4bc4-8a55-0469cbdc9510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:46.312081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9f4f-8d66-4bc4-8a55-0469cbdc9510 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:46.312365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9f4f-8d66-4bc4-8a55-0469cbdc9510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:46.312680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9f4f-8d66-4bc4-8a55-0469cbdc9510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:46.313333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb9f4f-8d66-4bc4-8a55-0469cbdc9510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:46.315014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb9f4f-8d66-4bc4-8a55-0469cbdc9510 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:46.327109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1207/2430] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:53:46.331526 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664703-559a-4f73-b2dd-b9d6e80a8625 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:46.342837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664703-559a-4f73-b2dd-b9d6e80a8625 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:46.347552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bea59-dd54-461e-a3b5-093dd05f3788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:46.348236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bea59-dd54-461e-a3b5-093dd05f3788 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:46.348542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bea59-dd54-461e-a3b5-093dd05f3788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:46.348820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bea59-dd54-461e-a3b5-093dd05f3788 admin admin] PATH_INFO: `/v3/projects/2cfb480658dd40f1a1f0fd5c627e843c/users/b5cf28697ff04a3ebd70b780c10dd830/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:46.367443 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4bea59-dd54-461e-a3b5-093dd05f3788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cfb480658dd40f1a1f0fd5c627e843c, user_id=b5cf28697ff04a3ebd70b780c10dd830, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:46.370254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4bea59-dd54-461e-a3b5-093dd05f3788 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:46.373670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:53:46.386318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1224/2431] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:53:46 2022] PUT /identity/v3/projects/2cfb480658dd40f1a1f0fd5c627e843c/users/b5cf28697ff04a3ebd70b780c10dd830/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:53:46.391492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14029c3e-8533-4b73-8707-a3327cb88dd1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:46.391997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14029c3e-8533-4b73-8707-a3327cb88dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:46.391997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14029c3e-8533-4b73-8707-a3327cb88dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:46.413200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-14029c3e-8533-4b73-8707-a3327cb88dd1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:53:46.414638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-14029c3e-8533-4b73-8707-a3327cb88dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:46.475493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1208/2432] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:53:46.485751 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bea59-dd54-461e-a3b5-093dd05f3788 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:46.487692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9f4bea59-dd54-461e-a3b5-093dd05f3788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:46.551045 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bea59-dd54-461e-a3b5-093dd05f3788 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:46.554713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8aeb3-6efc-4bd7-bfbc-4b3e7de7f476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:46.555318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8aeb3-6efc-4bd7-bfbc-4b3e7de7f476 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:46.555616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8aeb3-6efc-4bd7-bfbc-4b3e7de7f476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:46.555881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8aeb3-6efc-4bd7-bfbc-4b3e7de7f476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:46.566584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8aeb3-6efc-4bd7-bfbc-4b3e7de7f476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:46.568062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8aeb3-6efc-4bd7-bfbc-4b3e7de7f476 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:46.581076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1225/2433] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:53:49.523688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14029c3e-8533-4b73-8707-a3327cb88dd1 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:49.525628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-14029c3e-8533-4b73-8707-a3327cb88dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:49.547159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14029c3e-8533-4b73-8707-a3327cb88dd1 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:49.553359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b0bd3-797e-47d4-b0ae-865fb5fcf70c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:49.554459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b0bd3-797e-47d4-b0ae-865fb5fcf70c service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:49.554901 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b0bd3-797e-47d4-b0ae-865fb5fcf70c service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:49.555330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b0bd3-797e-47d4-b0ae-865fb5fcf70c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:49.558158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8d3b0bd3-797e-47d4-b0ae-865fb5fcf70c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:49.574184 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b0bd3-797e-47d4-b0ae-865fb5fcf70c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:49.577524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b0bd3-797e-47d4-b0ae-865fb5fcf70c service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:49.591262 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1209/2434] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:53:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:53:51.129765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1586-269b-4c2b-ba78-306caaaee6a2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:51.130584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1586-269b-4c2b-ba78-306caaaee6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:51.130928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1586-269b-4c2b-ba78-306caaaee6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:51.164858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-567c1586-269b-4c2b-ba78-306caaaee6a2 None None] MFA Rules not processed for user `b5cf28697ff04a3ebd70b780c10dd830`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:53:51.166112 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-567c1586-269b-4c2b-ba78-306caaaee6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:51.289583 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1226/2435] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:53:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:53:51.294656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b0bd3-797e-47d4-b0ae-865fb5fcf70c service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:51.304319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b0bd3-797e-47d4-b0ae-865fb5fcf70c service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:51.311837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807b831-40ae-4525-94b3-17f95cd486ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:51.312795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807b831-40ae-4525-94b3-17f95cd486ce admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:51.313263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807b831-40ae-4525-94b3-17f95cd486ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:51.313691 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807b831-40ae-4525-94b3-17f95cd486ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:51.314503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807b831-40ae-4525-94b3-17f95cd486ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:51.317016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807b831-40ae-4525-94b3-17f95cd486ce admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:51.353706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1210/2436] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:53:51 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:53:51.356075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1586-269b-4c2b-ba78-306caaaee6a2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:51.369790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1586-269b-4c2b-ba78-306caaaee6a2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:51.452347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b9a43-7e4c-47dd-88e7-d1d35e6328ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:51.453828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b9a43-7e4c-47dd-88e7-d1d35e6328ce admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:51.454118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b9a43-7e4c-47dd-88e7-d1d35e6328ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:51.454395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b9a43-7e4c-47dd-88e7-d1d35e6328ce admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:51.455099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b9a43-7e4c-47dd-88e7-d1d35e6328ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:51.458289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b9a43-7e4c-47dd-88e7-d1d35e6328ce admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:51.489950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1227/2437] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:53:51 2022] POST /identity/v3/users => generated 291 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:53:51.493681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807b831-40ae-4525-94b3-17f95cd486ce admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:51.504822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807b831-40ae-4525-94b3-17f95cd486ce admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:51.511795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbea049-0064-450d-baaf-c70d5818b1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:51.512085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbea049-0064-450d-baaf-c70d5818b1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:51.512460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbea049-0064-450d-baaf-c70d5818b1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:51.512931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbea049-0064-450d-baaf-c70d5818b1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:51.514077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbea049-0064-450d-baaf-c70d5818b1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:51.515921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbea049-0064-450d-baaf-c70d5818b1e6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:51.530038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1211/2438] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:53:51.535088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b9a43-7e4c-47dd-88e7-d1d35e6328ce admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:51.547133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b9a43-7e4c-47dd-88e7-d1d35e6328ce admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:51.552902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a742-36e8-4d26-b7c7-a76efa2a1c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:51.553716 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a742-36e8-4d26-b7c7-a76efa2a1c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:51.554095 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a742-36e8-4d26-b7c7-a76efa2a1c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:51.554467 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a742-36e8-4d26-b7c7-a76efa2a1c04 admin admin] PATH_INFO: `/v3/projects/16d704b0ba214a3d9d55260ec5e13993/users/ca01c59c7e70448fb7119347b2bb581d/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:51.581444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4a742-36e8-4d26-b7c7-a76efa2a1c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d704b0ba214a3d9d55260ec5e13993, user_id=ca01c59c7e70448fb7119347b2bb581d, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:51.584949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4a742-36e8-4d26-b7c7-a76efa2a1c04 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:51.591051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:53:51.605270 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1228/2439] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:53:51 2022] PUT /identity/v3/projects/16d704b0ba214a3d9d55260ec5e13993/users/ca01c59c7e70448fb7119347b2bb581d/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:53:51.609075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbea049-0064-450d-baaf-c70d5818b1e6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:51.622278 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbea049-0064-450d-baaf-c70d5818b1e6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:51.817026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a62bb-767d-48a7-8dbf-3d890de81106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:51.817672 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a62bb-767d-48a7-8dbf-3d890de81106 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:51.818140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a62bb-767d-48a7-8dbf-3d890de81106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:51.818575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a62bb-767d-48a7-8dbf-3d890de81106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:51.819690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a62bb-767d-48a7-8dbf-3d890de81106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:51.822085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a62bb-767d-48a7-8dbf-3d890de81106 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:51.844739 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1212/2440] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:53:51.848929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a742-36e8-4d26-b7c7-a76efa2a1c04 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:51.863544 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a742-36e8-4d26-b7c7-a76efa2a1c04 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:51.868992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbc3b7-e6d4-45e9-9445-d2f1538c1ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:51.870169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbc3b7-e6d4-45e9-9445-d2f1538c1ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:51.870630 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbc3b7-e6d4-45e9-9445-d2f1538c1ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:51.871062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbc3b7-e6d4-45e9-9445-d2f1538c1ed2 admin admin] PATH_INFO: `/v3/projects/16d704b0ba214a3d9d55260ec5e13993/users/ca01c59c7e70448fb7119347b2bb581d/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:51.873839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbc3b7-e6d4-45e9-9445-d2f1538c1ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d704b0ba214a3d9d55260ec5e13993, user_id=ca01c59c7e70448fb7119347b2bb581d, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:51.878679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbc3b7-e6d4-45e9-9445-d2f1538c1ed2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:51.892553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1229/2441] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:53:51 2022] PUT /identity/v3/projects/16d704b0ba214a3d9d55260ec5e13993/users/ca01c59c7e70448fb7119347b2bb581d/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:53:54.762460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a62bb-767d-48a7-8dbf-3d890de81106 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:54.765508 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-af1a62bb-767d-48a7-8dbf-3d890de81106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:54.785581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a62bb-767d-48a7-8dbf-3d890de81106 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:54.818253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:53:54.867752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29fb42-0870-467f-9e10-fb271b568691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:54.868400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29fb42-0870-467f-9e10-fb271b568691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:54.868725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29fb42-0870-467f-9e10-fb271b568691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:54.869006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29fb42-0870-467f-9e10-fb271b568691 admin admin] PATH_INFO: `/v3/users/38c4d208994a40fe82310adb18bb3547` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:54.898480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29fb42-0870-467f-9e10-fb271b568691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c4d208994a40fe82310adb18bb3547)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:54.902124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29fb42-0870-467f-9e10-fb271b568691 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:54.985243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fe29fb42-0870-467f-9e10-fb271b568691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c4d208994a40fe82310adb18bb3547'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:54.986103 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fe29fb42-0870-467f-9e10-fb271b568691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c4d208994a40fe82310adb18bb3547'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:55.002445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fe29fb42-0870-467f-9e10-fb271b568691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c4d208994a40fe82310adb18bb3547'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:55.015442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fe29fb42-0870-467f-9e10-fb271b568691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c4d208994a40fe82310adb18bb3547'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:55.016925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fe29fb42-0870-467f-9e10-fb271b568691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c4d208994a40fe82310adb18bb3547'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:55.051001 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1213/2442] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:53:54 2022] DELETE /identity/v3/users/38c4d208994a40fe82310adb18bb3547 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:53:55.330359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbc3b7-e6d4-45e9-9445-d2f1538c1ed2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:55.332189 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0abbc3b7-e6d4-45e9-9445-d2f1538c1ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:55.349790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbc3b7-e6d4-45e9-9445-d2f1538c1ed2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:55.458784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:55.459512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:55.459862 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:55.460210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] PATH_INFO: `/v3/projects/aacdfff11f2a41f889f7d7cdc41c12dc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:55.477192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aacdfff11f2a41f889f7d7cdc41c12dc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:55.479966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:55.610957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aacdfff11f2a41f889f7d7cdc41c12dc', 'request_id': 'req-69cb0781-028d-4e8c-a080-cb9193d96a63'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:55.622591 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aacdfff11f2a41f889f7d7cdc41c12dc', 'request_id': 'req-69cb0781-028d-4e8c-a080-cb9193d96a63'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:55.631825 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aacdfff11f2a41f889f7d7cdc41c12dc', 'request_id': 'req-69cb0781-028d-4e8c-a080-cb9193d96a63'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:55.664452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] The token cache is being invalidate because project aacdfff11f2a41f889f7d7cdc41c12dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:53:55.664765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:55.665477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cb0781-028d-4e8c-a080-cb9193d96a63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:53:55.668091 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1230/2443] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:53:55 2022] DELETE /identity/v3/projects/aacdfff11f2a41f889f7d7cdc41c12dc => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:53:55.672627 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af1bbc-3d1b-420b-878e-4134a6b9701c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:55.673619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af1bbc-3d1b-420b-878e-4134a6b9701c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:55.674158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af1bbc-3d1b-420b-878e-4134a6b9701c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:55.718604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b0af1bbc-3d1b-420b-878e-4134a6b9701c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:53:55.720477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b0af1bbc-3d1b-420b-878e-4134a6b9701c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:55.804753 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1214/2444] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:53:55.814203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ade77-3b39-411a-943b-07c3d88c357d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:55.814531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ade77-3b39-411a-943b-07c3d88c357d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:55.814815 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ade77-3b39-411a-943b-07c3d88c357d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:55.845061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-848ade77-3b39-411a-943b-07c3d88c357d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:53:55.846663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-848ade77-3b39-411a-943b-07c3d88c357d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:55.864806 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1231/2445] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:53:55.874449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af1bbc-3d1b-420b-878e-4134a6b9701c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:55.890159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af1bbc-3d1b-420b-878e-4134a6b9701c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:55.907344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e25570-3215-4a9b-88dc-8c8083db62d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:55.909249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25570-3215-4a9b-88dc-8c8083db62d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:55.909715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25570-3215-4a9b-88dc-8c8083db62d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:55.912982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25570-3215-4a9b-88dc-8c8083db62d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:55.913913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e25570-3215-4a9b-88dc-8c8083db62d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:53:55.917956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e25570-3215-4a9b-88dc-8c8083db62d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:55.922562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e25570-3215-4a9b-88dc-8c8083db62d1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:55.952891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1215/2446] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:53:55.954295 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ade77-3b39-411a-943b-07c3d88c357d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:55.968950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ade77-3b39-411a-943b-07c3d88c357d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:55.977051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82670c6e-a08a-4b98-a1a8-3540162d8f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:55.977988 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82670c6e-a08a-4b98-a1a8-3540162d8f95 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:55.978453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82670c6e-a08a-4b98-a1a8-3540162d8f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:55.979095 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82670c6e-a08a-4b98-a1a8-3540162d8f95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:55.980662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82670c6e-a08a-4b98-a1a8-3540162d8f95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:55.983109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82670c6e-a08a-4b98-a1a8-3540162d8f95 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:56.041193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1232/2447] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:53:55 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:53:56.047748 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e25570-3215-4a9b-88dc-8c8083db62d1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:56.060102 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e25570-3215-4a9b-88dc-8c8083db62d1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:56.096997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:53:56.200364 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ef63d-60f7-4bbf-831b-e4773002a48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:56.201507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ef63d-60f7-4bbf-831b-e4773002a48f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:56.202131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ef63d-60f7-4bbf-831b-e4773002a48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:56.202720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ef63d-60f7-4bbf-831b-e4773002a48f admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:56.203791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ef63d-60f7-4bbf-831b-e4773002a48f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:56.206129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ef63d-60f7-4bbf-831b-e4773002a48f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:56.248112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1216/2448] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:53:56 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:53:56.252414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82670c6e-a08a-4b98-a1a8-3540162d8f95 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:56.266414 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82670c6e-a08a-4b98-a1a8-3540162d8f95 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:56.276675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2973d7-b236-4b9c-9e02-e71b7fe6d9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:56.277621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2973d7-b236-4b9c-9e02-e71b7fe6d9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:56.278005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2973d7-b236-4b9c-9e02-e71b7fe6d9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:56.278636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2973d7-b236-4b9c-9e02-e71b7fe6d9f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:56.279462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2973d7-b236-4b9c-9e02-e71b7fe6d9f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:56.281804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2973d7-b236-4b9c-9e02-e71b7fe6d9f1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:56.305489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1233/2449] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:53:56.306158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ef63d-60f7-4bbf-831b-e4773002a48f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:56.318725 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ef63d-60f7-4bbf-831b-e4773002a48f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:56.325806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14ae77-084d-493f-ac8d-22631ef1c5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:56.326445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14ae77-084d-493f-ac8d-22631ef1c5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:56.326731 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14ae77-084d-493f-ac8d-22631ef1c5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:56.327014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14ae77-084d-493f-ac8d-22631ef1c5f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:56.328020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14ae77-084d-493f-ac8d-22631ef1c5f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:56.329890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14ae77-084d-493f-ac8d-22631ef1c5f6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:56.350968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1217/2450] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:53:56.352658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2973d7-b236-4b9c-9e02-e71b7fe6d9f1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:56.364325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2973d7-b236-4b9c-9e02-e71b7fe6d9f1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:56.371897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0f111-1cc1-4993-865c-61e9f6a016d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:56.373627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0f111-1cc1-4993-865c-61e9f6a016d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:56.374122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0f111-1cc1-4993-865c-61e9f6a016d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:56.374591 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0f111-1cc1-4993-865c-61e9f6a016d4 admin admin] PATH_INFO: `/v3/projects/258a5512809f4fd29ff55b6c4a570cbc/users/85e3a6000e9341fdb8062cde59693480/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:56.396691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0f111-1cc1-4993-865c-61e9f6a016d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=258a5512809f4fd29ff55b6c4a570cbc, user_id=85e3a6000e9341fdb8062cde59693480, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:56.399810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0f111-1cc1-4993-865c-61e9f6a016d4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:56.413352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:53:56.443540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1234/2451] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:53:56 2022] PUT /identity/v3/projects/258a5512809f4fd29ff55b6c4a570cbc/users/85e3a6000e9341fdb8062cde59693480/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:53:56.448835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b65b-762e-4d68-a4b1-e62e305caecf None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:56.449332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b65b-762e-4d68-a4b1-e62e305caecf None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:56.449710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b65b-762e-4d68-a4b1-e62e305caecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:56.487233 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d782b65b-762e-4d68-a4b1-e62e305caecf None None] MFA Rules not processed for user `85e3a6000e9341fdb8062cde59693480`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:53:56.487233 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d782b65b-762e-4d68-a4b1-e62e305caecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:56.590368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1218/2452] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:53:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:53:56.602310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0f111-1cc1-4993-865c-61e9f6a016d4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:56.604885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1de0f111-1cc1-4993-865c-61e9f6a016d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:56.725320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0f111-1cc1-4993-865c-61e9f6a016d4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:56.733131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32853257-102f-41f4-9ff7-9af52b84206f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:56.734450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32853257-102f-41f4-9ff7-9af52b84206f service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:56.734887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32853257-102f-41f4-9ff7-9af52b84206f service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:56.735345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32853257-102f-41f4-9ff7-9af52b84206f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:56.750592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32853257-102f-41f4-9ff7-9af52b84206f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:56.753831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32853257-102f-41f4-9ff7-9af52b84206f service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:56.778345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1235/2453] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:53:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:53:58.636975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49afd9f-f599-44cc-9714-2b1806d5e554 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:58.637541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49afd9f-f599-44cc-9714-2b1806d5e554 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:58.637541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49afd9f-f599-44cc-9714-2b1806d5e554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:58.690983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d49afd9f-f599-44cc-9714-2b1806d5e554 None None] MFA Rules not processed for user `ca01c59c7e70448fb7119347b2bb581d`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:53:58.692548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d49afd9f-f599-44cc-9714-2b1806d5e554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:58.806139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1219/2454] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:53:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:53:58.821334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32853257-102f-41f4-9ff7-9af52b84206f service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:53:58.823184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-32853257-102f-41f4-9ff7-9af52b84206f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:58.845516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32853257-102f-41f4-9ff7-9af52b84206f service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:53:58.852872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273da03-9271-48ad-bff4-a3cbb449305e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:53:58.853497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273da03-9271-48ad-bff4-a3cbb449305e service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:53:58.854326 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273da03-9271-48ad-bff4-a3cbb449305e service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:53:58.854711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273da03-9271-48ad-bff4-a3cbb449305e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:53:58.862208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f273da03-9271-48ad-bff4-a3cbb449305e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:53:58.940648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273da03-9271-48ad-bff4-a3cbb449305e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:53:58.942599 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273da03-9271-48ad-bff4-a3cbb449305e service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:53:58.954554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1236/2455] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:53:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:00.001167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49afd9f-f599-44cc-9714-2b1806d5e554 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:00.018598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:00.072143 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d16d6-2693-4c5b-bbe8-7cd85344bb2b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:00.072881 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d16d6-2693-4c5b-bbe8-7cd85344bb2b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:00.073081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d16d6-2693-4c5b-bbe8-7cd85344bb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:00.082845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49afd9f-f599-44cc-9714-2b1806d5e554 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:00.087995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:00.088946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:00.089532 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:00.089815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] PATH_INFO: `/v3/users/85e3a6000e9341fdb8062cde59693480` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:00.091183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e3a6000e9341fdb8062cde59693480)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:00.092946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-975d16d6-2693-4c5b-bbe8-7cd85344bb2b None None] MFA Rules not processed for user `b5cf28697ff04a3ebd70b780c10dd830`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:00.093203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:00.094165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-975d16d6-2693-4c5b-bbe8-7cd85344bb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:00.101761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1237/2456] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:00.155084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e3a6000e9341fdb8062cde59693480'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:00.155831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e3a6000e9341fdb8062cde59693480'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:00.163890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e3a6000e9341fdb8062cde59693480'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:00.175172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e3a6000e9341fdb8062cde59693480'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:00.175937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e3a6000e9341fdb8062cde59693480'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:00.207215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3f5d5-e17f-48c9-832d-de454ce0d44a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:00.207733 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3f5d5-e17f-48c9-832d-de454ce0d44a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:00.208130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3f5d5-e17f-48c9-832d-de454ce0d44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:00.208585 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1220/2457] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:53:59 2022] DELETE /identity/v3/users/85e3a6000e9341fdb8062cde59693480 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:00.230734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-43a3f5d5-e17f-48c9-832d-de454ce0d44a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:00.232203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-43a3f5d5-e17f-48c9-832d-de454ce0d44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:00.316383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1238/2458] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:00.327966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:00.329719 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:00.404952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257adcb-6a3b-4e3d-883d-c14090336752 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:00.409511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e803d-2183-466d-a739-76bc51715458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:00.410144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e803d-2183-466d-a739-76bc51715458 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:00.410601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e803d-2183-466d-a739-76bc51715458 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:00.410888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e803d-2183-466d-a739-76bc51715458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:00.422921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e803d-2183-466d-a739-76bc51715458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:00.424401 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e803d-2183-466d-a739-76bc51715458 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:00.436558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1221/2459] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:00.828449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3f5d5-e17f-48c9-832d-de454ce0d44a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:00.829915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-43a3f5d5-e17f-48c9-832d-de454ce0d44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:00.853119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3f5d5-e17f-48c9-832d-de454ce0d44a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:00.862320 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:00.863146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:00.863435 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:00.863939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] PATH_INFO: `/v3/projects/258a5512809f4fd29ff55b6c4a570cbc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:00.865250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=258a5512809f4fd29ff55b6c4a570cbc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:00.867235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:00.882673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:00.944236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '258a5512809f4fd29ff55b6c4a570cbc', 'request_id': 'req-c6184049-901f-441f-a417-eb1a04b104b2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:00.953306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '258a5512809f4fd29ff55b6c4a570cbc', 'request_id': 'req-c6184049-901f-441f-a417-eb1a04b104b2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:00.961883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '258a5512809f4fd29ff55b6c4a570cbc', 'request_id': 'req-c6184049-901f-441f-a417-eb1a04b104b2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:00.979027 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] The token cache is being invalidate because project 258a5512809f4fd29ff55b6c4a570cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:00.979683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:00.981331 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6184049-901f-441f-a417-eb1a04b104b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:00.983178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1239/2460] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:00 2022] DELETE /identity/v3/projects/258a5512809f4fd29ff55b6c4a570cbc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:00.988087 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bcca8-2ce7-4a02-9a10-21a17a867bfd None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:00.988729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bcca8-2ce7-4a02-9a10-21a17a867bfd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:00.989188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bcca8-2ce7-4a02-9a10-21a17a867bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.019810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c94bcca8-2ce7-4a02-9a10-21a17a867bfd None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:01.021439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c94bcca8-2ce7-4a02-9a10-21a17a867bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:01.065557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:01.109151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1222/2461] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:01.117168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49908afa-7112-45ee-acd1-4bd541493b48 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:01.117666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49908afa-7112-45ee-acd1-4bd541493b48 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:01.117766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49908afa-7112-45ee-acd1-4bd541493b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.144309 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-49908afa-7112-45ee-acd1-4bd541493b48 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:01.145574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-49908afa-7112-45ee-acd1-4bd541493b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:01.153910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1240/2462] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:01.158069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bcca8-2ce7-4a02-9a10-21a17a867bfd None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:01.168598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bcca8-2ce7-4a02-9a10-21a17a867bfd None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:01.174542 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd53d3b-fd08-4fea-b140-0539c1387350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:01.175257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd53d3b-fd08-4fea-b140-0539c1387350 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:01.175604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd53d3b-fd08-4fea-b140-0539c1387350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:01.176085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd53d3b-fd08-4fea-b140-0539c1387350 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.176971 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd53d3b-fd08-4fea-b140-0539c1387350 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:54:01.177275 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd53d3b-fd08-4fea-b140-0539c1387350 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:01.178889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd53d3b-fd08-4fea-b140-0539c1387350 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:01.192020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1223/2463] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:54:01.196930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49908afa-7112-45ee-acd1-4bd541493b48 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:01.212003 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49908afa-7112-45ee-acd1-4bd541493b48 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:01.216182 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c0f87-6631-47f7-80eb-5abea7e80502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:01.216841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c0f87-6631-47f7-80eb-5abea7e80502 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:01.217130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c0f87-6631-47f7-80eb-5abea7e80502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:01.217400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c0f87-6631-47f7-80eb-5abea7e80502 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.218044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c0f87-6631-47f7-80eb-5abea7e80502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:01.219987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c0f87-6631-47f7-80eb-5abea7e80502 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:01.256782 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1241/2464] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:01 2022] POST /identity/v3/projects => generated 403 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:01.261066 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd53d3b-fd08-4fea-b140-0539c1387350 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:01.272036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd53d3b-fd08-4fea-b140-0539c1387350 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:01.342218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036fda1-e794-41ae-af7e-f803582a9ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:01.342840 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036fda1-e794-41ae-af7e-f803582a9ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:01.343121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036fda1-e794-41ae-af7e-f803582a9ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:01.343396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036fda1-e794-41ae-af7e-f803582a9ec8 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.344051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036fda1-e794-41ae-af7e-f803582a9ec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:01.346056 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036fda1-e794-41ae-af7e-f803582a9ec8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:01.371872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1224/2465] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:01 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:01.376069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c0f87-6631-47f7-80eb-5abea7e80502 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:01.387341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c0f87-6631-47f7-80eb-5abea7e80502 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:01.391959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6b926-2a8c-4a68-9120-7afdad5893cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:01.392597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6b926-2a8c-4a68-9120-7afdad5893cf admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:01.392882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6b926-2a8c-4a68-9120-7afdad5893cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:01.393154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6b926-2a8c-4a68-9120-7afdad5893cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.393798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6b926-2a8c-4a68-9120-7afdad5893cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:01.395394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6b926-2a8c-4a68-9120-7afdad5893cf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:01.411989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1242/2466] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:01.413283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036fda1-e794-41ae-af7e-f803582a9ec8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:01.424905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036fda1-e794-41ae-af7e-f803582a9ec8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:01.429749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447b08e-510b-478b-8c59-e0e6bfe8595d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:01.430557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447b08e-510b-478b-8c59-e0e6bfe8595d admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:01.430844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447b08e-510b-478b-8c59-e0e6bfe8595d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:01.431116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447b08e-510b-478b-8c59-e0e6bfe8595d admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.431884 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447b08e-510b-478b-8c59-e0e6bfe8595d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:01.433589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447b08e-510b-478b-8c59-e0e6bfe8595d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:01.445402 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1225/2467] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:54:01.449996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6b926-2a8c-4a68-9120-7afdad5893cf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:01.461885 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6b926-2a8c-4a68-9120-7afdad5893cf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:01.467724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d8da5-13f8-4e09-a54f-84b34bb6ea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:01.468402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d8da5-13f8-4e09-a54f-84b34bb6ea37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:01.468715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d8da5-13f8-4e09-a54f-84b34bb6ea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:01.468993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d8da5-13f8-4e09-a54f-84b34bb6ea37 admin admin] PATH_INFO: `/v3/projects/9753bbb5276f4ceaac02599b868b3079/users/ae618ebcb24b4df5a943500c4b026b5b/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.485944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d8da5-13f8-4e09-a54f-84b34bb6ea37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9753bbb5276f4ceaac02599b868b3079, user_id=ae618ebcb24b4df5a943500c4b026b5b, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:01.489645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d8da5-13f8-4e09-a54f-84b34bb6ea37 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:01.509425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1243/2468] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:01 2022] PUT /identity/v3/projects/9753bbb5276f4ceaac02599b868b3079/users/ae618ebcb24b4df5a943500c4b026b5b/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:01.522635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87439f2-77d6-4a30-a51f-8d38c286bfce None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:01.522853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87439f2-77d6-4a30-a51f-8d38c286bfce None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:01.523133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87439f2-77d6-4a30-a51f-8d38c286bfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.554661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d87439f2-77d6-4a30-a51f-8d38c286bfce None None] MFA Rules not processed for user `ae618ebcb24b4df5a943500c4b026b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:01.555942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d87439f2-77d6-4a30-a51f-8d38c286bfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:01.645493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1226/2469] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:01 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:01.653323 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d8da5-13f8-4e09-a54f-84b34bb6ea37 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:01.741764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d8da5-13f8-4e09-a54f-84b34bb6ea37 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:01.747431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594eb58-cecc-4451-8b40-690f5aa2d2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:01.748063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594eb58-cecc-4451-8b40-690f5aa2d2ae admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:01.748346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594eb58-cecc-4451-8b40-690f5aa2d2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:01.748658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594eb58-cecc-4451-8b40-690f5aa2d2ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.749437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594eb58-cecc-4451-8b40-690f5aa2d2ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:01.754049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594eb58-cecc-4451-8b40-690f5aa2d2ae admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:01.783076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1244/2470] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:01 2022] POST /identity/v3/projects => generated 405 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:01.787440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87439f2-77d6-4a30-a51f-8d38c286bfce None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:01.801647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87439f2-77d6-4a30-a51f-8d38c286bfce None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:01.840856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:01.895479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6593171-1da9-47a9-aaca-3c3ca58c91bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:01.898059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6593171-1da9-47a9-aaca-3c3ca58c91bd admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:01.898391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6593171-1da9-47a9-aaca-3c3ca58c91bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:01.898710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6593171-1da9-47a9-aaca-3c3ca58c91bd admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.899819 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6593171-1da9-47a9-aaca-3c3ca58c91bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:01.902733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6593171-1da9-47a9-aaca-3c3ca58c91bd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:01.939353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1227/2471] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:01 2022] POST /identity/v3/users => generated 309 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:54:01.943380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594eb58-cecc-4451-8b40-690f5aa2d2ae admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:01.959738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594eb58-cecc-4451-8b40-690f5aa2d2ae admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:01.966605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd3b45-3939-4abb-9f92-a9ec8079ebdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:01.967304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd3b45-3939-4abb-9f92-a9ec8079ebdb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:01.967659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd3b45-3939-4abb-9f92-a9ec8079ebdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:01.968007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd3b45-3939-4abb-9f92-a9ec8079ebdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:01.968774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd3b45-3939-4abb-9f92-a9ec8079ebdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:01.970499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd3b45-3939-4abb-9f92-a9ec8079ebdb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:01.988412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1245/2472] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:01.990919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6593171-1da9-47a9-aaca-3c3ca58c91bd admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:02.005623 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6593171-1da9-47a9-aaca-3c3ca58c91bd admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:02.014836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa7b70-c95f-4f1d-87c8-6bfe5cb61549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:02.017110 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa7b70-c95f-4f1d-87c8-6bfe5cb61549 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:02.017403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa7b70-c95f-4f1d-87c8-6bfe5cb61549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:02.018047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa7b70-c95f-4f1d-87c8-6bfe5cb61549 admin admin] PATH_INFO: `/v3/projects/c417f9ab57c8486d81fab760b7c83db7/users/864c1fe9a9764aad9412662820146abc/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:02.045479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa7b70-c95f-4f1d-87c8-6bfe5cb61549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c417f9ab57c8486d81fab760b7c83db7, user_id=864c1fe9a9764aad9412662820146abc, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:02.049183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa7b70-c95f-4f1d-87c8-6bfe5cb61549 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:02.066229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1228/2473] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:01 2022] PUT /identity/v3/projects/c417f9ab57c8486d81fab760b7c83db7/users/864c1fe9a9764aad9412662820146abc/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:02.070823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd3b45-3939-4abb-9f92-a9ec8079ebdb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:02.084647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd3b45-3939-4abb-9f92-a9ec8079ebdb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:02.094291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:02.171559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26458d3d-a3a2-49d0-9291-ffbca54c64a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:02.172601 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26458d3d-a3a2-49d0-9291-ffbca54c64a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:02.172932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26458d3d-a3a2-49d0-9291-ffbca54c64a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:02.173258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26458d3d-a3a2-49d0-9291-ffbca54c64a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:02.174137 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26458d3d-a3a2-49d0-9291-ffbca54c64a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:02.175833 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26458d3d-a3a2-49d0-9291-ffbca54c64a7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:02.192646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1246/2474] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:02.194243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa7b70-c95f-4f1d-87c8-6bfe5cb61549 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:02.209408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa7b70-c95f-4f1d-87c8-6bfe5cb61549 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:02.216115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5a549-494a-4184-8d48-bd3c90a60527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:02.217011 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5a549-494a-4184-8d48-bd3c90a60527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:02.217300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5a549-494a-4184-8d48-bd3c90a60527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:02.217572 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5a549-494a-4184-8d48-bd3c90a60527 admin admin] PATH_INFO: `/v3/projects/c417f9ab57c8486d81fab760b7c83db7/users/864c1fe9a9764aad9412662820146abc/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:02.219708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5a549-494a-4184-8d48-bd3c90a60527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c417f9ab57c8486d81fab760b7c83db7, user_id=864c1fe9a9764aad9412662820146abc, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:02.222701 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5a549-494a-4184-8d48-bd3c90a60527 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:02.236242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1229/2475] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:02 2022] PUT /identity/v3/projects/c417f9ab57c8486d81fab760b7c83db7/users/864c1fe9a9764aad9412662820146abc/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:02.250214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ceed2-5a5c-4ed2-a250-2f699d02336c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:02.250610 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ceed2-5a5c-4ed2-a250-2f699d02336c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:02.250826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ceed2-5a5c-4ed2-a250-2f699d02336c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:02.292758 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ed2ceed2-5a5c-4ed2-a250-2f699d02336c None None] MFA Rules not processed for user `864c1fe9a9764aad9412662820146abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:02.294073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ed2ceed2-5a5c-4ed2-a250-2f699d02336c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:02.407081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1247/2476] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:02.419624 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5a549-494a-4184-8d48-bd3c90a60527 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:02.422047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a9f5a549-494a-4184-8d48-bd3c90a60527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:02.506391 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5a549-494a-4184-8d48-bd3c90a60527 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:02.513210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9401d-a6db-4b35-bedc-fd78bfcb8271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:02.514126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9401d-a6db-4b35-bedc-fd78bfcb8271 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:02.514575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9401d-a6db-4b35-bedc-fd78bfcb8271 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:02.514986 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9401d-a6db-4b35-bedc-fd78bfcb8271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:02.528647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9401d-a6db-4b35-bedc-fd78bfcb8271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:02.530253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9401d-a6db-4b35-bedc-fd78bfcb8271 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:02.543593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1230/2477] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:02 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:54:02.706210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ceed2-5a5c-4ed2-a250-2f699d02336c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:02.777904 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ceed2-5a5c-4ed2-a250-2f699d02336c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:02.782436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:02.783125 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:02.783405 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:02.783676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] PATH_INFO: `/v3/users/ae618ebcb24b4df5a943500c4b026b5b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:02.784827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae618ebcb24b4df5a943500c4b026b5b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:02.786750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:02.827032 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:02.839793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae618ebcb24b4df5a943500c4b026b5b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:02.846651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae618ebcb24b4df5a943500c4b026b5b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:02.847321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae618ebcb24b4df5a943500c4b026b5b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:02.847925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae618ebcb24b4df5a943500c4b026b5b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:02.856947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae618ebcb24b4df5a943500c4b026b5b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:02.878133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1248/2478] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:54:02 2022] DELETE /identity/v3/users/ae618ebcb24b4df5a943500c4b026b5b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:02.882557 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68374e-0760-46b9-a6e1-cf920f8a1a26 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:02.882868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68374e-0760-46b9-a6e1-cf920f8a1a26 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:02.883135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68374e-0760-46b9-a6e1-cf920f8a1a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:02.905253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7d68374e-0760-46b9-a6e1-cf920f8a1a26 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:02.906518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7d68374e-0760-46b9-a6e1-cf920f8a1a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:02.973947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1231/2479] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:02.984178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:02.985686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:03.049217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313f13d-8b26-4ea5-9092-eae22f261d88 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:03.052899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff4893-0564-499d-bf9a-3345c736cdbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:03.053508 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff4893-0564-499d-bf9a-3345c736cdbe service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:03.053789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff4893-0564-499d-bf9a-3345c736cdbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:03.054061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff4893-0564-499d-bf9a-3345c736cdbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:03.064144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff4893-0564-499d-bf9a-3345c736cdbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:03.065616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff4893-0564-499d-bf9a-3345c736cdbe service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:03.076590 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1249/2480] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:03.383210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68374e-0760-46b9-a6e1-cf920f8a1a26 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:03.385187 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7d68374e-0760-46b9-a6e1-cf920f8a1a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:03.397737 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68374e-0760-46b9-a6e1-cf920f8a1a26 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:03.404367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:03.405062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:03.405342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:03.405617 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] PATH_INFO: `/v3/projects/9753bbb5276f4ceaac02599b868b3079` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:03.406750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9753bbb5276f4ceaac02599b868b3079)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:03.408889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:03.433021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:03.462200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9753bbb5276f4ceaac02599b868b3079', 'request_id': 'req-078b72d9-8c48-4a22-af8c-806b4677af1e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:03.468338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9753bbb5276f4ceaac02599b868b3079', 'request_id': 'req-078b72d9-8c48-4a22-af8c-806b4677af1e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:03.474745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9753bbb5276f4ceaac02599b868b3079', 'request_id': 'req-078b72d9-8c48-4a22-af8c-806b4677af1e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:03.489069 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] The token cache is being invalidate because project 9753bbb5276f4ceaac02599b868b3079 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:03.489432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:03.490095 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:03.492062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1232/2481] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:03 2022] DELETE /identity/v3/projects/9753bbb5276f4ceaac02599b868b3079 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:03.496681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff4893-0564-499d-bf9a-3345c736cdbe service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:03.506456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff4893-0564-499d-bf9a-3345c736cdbe service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:03.568183 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2232-085a-438c-8d9c-9040c1a3a9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:03.569104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2232-085a-438c-8d9c-9040c1a3a9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:03.569384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2232-085a-438c-8d9c-9040c1a3a9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:03.569660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2232-085a-438c-8d9c-9040c1a3a9ed admin admin] PATH_INFO: `/v3/users/864c1fe9a9764aad9412662820146abc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:03.570785 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2232-085a-438c-8d9c-9040c1a3a9ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=864c1fe9a9764aad9412662820146abc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:03.573391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2232-085a-438c-8d9c-9040c1a3a9ed admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:03.625367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-479f2232-085a-438c-8d9c-9040c1a3a9ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '864c1fe9a9764aad9412662820146abc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:03.631249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:03.633457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-479f2232-085a-438c-8d9c-9040c1a3a9ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '864c1fe9a9764aad9412662820146abc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:03.634402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-479f2232-085a-438c-8d9c-9040c1a3a9ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '864c1fe9a9764aad9412662820146abc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:03.636154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-479f2232-085a-438c-8d9c-9040c1a3a9ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '864c1fe9a9764aad9412662820146abc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:03.643874 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-479f2232-085a-438c-8d9c-9040c1a3a9ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '864c1fe9a9764aad9412662820146abc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:03.662816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1250/2482] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:54:03 2022] DELETE /identity/v3/users/864c1fe9a9764aad9412662820146abc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:03.968014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:03.969900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:03.986219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b72d9-8c48-4a22-af8c-806b4677af1e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:04.065225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:04.065869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:04.066177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:04.066496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] PATH_INFO: `/v3/projects/c417f9ab57c8486d81fab760b7c83db7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:04.067651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c417f9ab57c8486d81fab760b7c83db7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:04.069990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:04.122118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c417f9ab57c8486d81fab760b7c83db7', 'request_id': 'req-5bc6ed70-e43b-4056-a354-17c121c1bea3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:04.128673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c417f9ab57c8486d81fab760b7c83db7', 'request_id': 'req-5bc6ed70-e43b-4056-a354-17c121c1bea3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:04.138618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c417f9ab57c8486d81fab760b7c83db7', 'request_id': 'req-5bc6ed70-e43b-4056-a354-17c121c1bea3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:04.152576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] The token cache is being invalidate because project c417f9ab57c8486d81fab760b7c83db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:04.152853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:04.153498 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5bc6ed70-e43b-4056-a354-17c121c1bea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:04.155003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1233/2483] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:03 2022] DELETE /identity/v3/projects/c417f9ab57c8486d81fab760b7c83db7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:04.160015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b28304-b314-4a53-a59b-147d336c0d7a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:04.160306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b28304-b314-4a53-a59b-147d336c0d7a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:04.160625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b28304-b314-4a53-a59b-147d336c0d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:04.184638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a0b28304-b314-4a53-a59b-147d336c0d7a None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:04.185867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a0b28304-b314-4a53-a59b-147d336c0d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:04.375504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1251/2484] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:04.383285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008d837-1816-433b-9aa3-911308fec81b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:04.383904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008d837-1816-433b-9aa3-911308fec81b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:04.383904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008d837-1816-433b-9aa3-911308fec81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:04.404742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2008d837-1816-433b-9aa3-911308fec81b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:04.405892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2008d837-1816-433b-9aa3-911308fec81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:04.413909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1234/2485] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:04.419094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b28304-b314-4a53-a59b-147d336c0d7a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:04.429804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b28304-b314-4a53-a59b-147d336c0d7a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:04.434956 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa13567-9608-4103-a1ef-6984885e63af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:04.435563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13567-9608-4103-a1ef-6984885e63af admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:04.435844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13567-9608-4103-a1ef-6984885e63af admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:04.436119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13567-9608-4103-a1ef-6984885e63af admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:04.436819 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa13567-9608-4103-a1ef-6984885e63af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:54:04.437122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa13567-9608-4103-a1ef-6984885e63af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:04.438754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa13567-9608-4103-a1ef-6984885e63af admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:04.451970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1252/2486] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:54:04.456423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008d837-1816-433b-9aa3-911308fec81b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:04.467180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008d837-1816-433b-9aa3-911308fec81b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:04.472526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e308c71-1e9d-441e-a888-508adf725577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:04.473174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e308c71-1e9d-441e-a888-508adf725577 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:04.473455 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e308c71-1e9d-441e-a888-508adf725577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:04.473727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e308c71-1e9d-441e-a888-508adf725577 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:04.474371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e308c71-1e9d-441e-a888-508adf725577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:04.476301 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e308c71-1e9d-441e-a888-508adf725577 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:04.507066 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1235/2487] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:04 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:04.510805 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa13567-9608-4103-a1ef-6984885e63af admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:04.521851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa13567-9608-4103-a1ef-6984885e63af admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:04.584708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284c497-5d2a-4058-8606-28e5fb78ec9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:04.585317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284c497-5d2a-4058-8606-28e5fb78ec9d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:04.585597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284c497-5d2a-4058-8606-28e5fb78ec9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:04.585867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284c497-5d2a-4058-8606-28e5fb78ec9d admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:04.586503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284c497-5d2a-4058-8606-28e5fb78ec9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:04.589128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284c497-5d2a-4058-8606-28e5fb78ec9d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:04.611737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1253/2488] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:04 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:54:04.616209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e308c71-1e9d-441e-a888-508adf725577 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:04.626620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e308c71-1e9d-441e-a888-508adf725577 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:04.631217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58ea86-f4df-49a6-ac52-ab17f1d9ab5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:04.631812 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58ea86-f4df-49a6-ac52-ab17f1d9ab5d admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:04.632094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58ea86-f4df-49a6-ac52-ab17f1d9ab5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:04.632367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58ea86-f4df-49a6-ac52-ab17f1d9ab5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:04.633060 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58ea86-f4df-49a6-ac52-ab17f1d9ab5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:04.635206 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58ea86-f4df-49a6-ac52-ab17f1d9ab5d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:04.638126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:04.648327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1236/2489] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:04.653265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284c497-5d2a-4058-8606-28e5fb78ec9d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:04.663734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284c497-5d2a-4058-8606-28e5fb78ec9d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:04.668640 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59450832-cab6-40ea-b432-12ad04698860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:04.669243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59450832-cab6-40ea-b432-12ad04698860 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:04.669525 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59450832-cab6-40ea-b432-12ad04698860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:04.669780 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59450832-cab6-40ea-b432-12ad04698860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:04.670568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59450832-cab6-40ea-b432-12ad04698860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:04.672939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59450832-cab6-40ea-b432-12ad04698860 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:04.690368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1254/2490] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:04.694855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58ea86-f4df-49a6-ac52-ab17f1d9ab5d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:04.705712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58ea86-f4df-49a6-ac52-ab17f1d9ab5d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:04.710618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8068812-c3ac-4bad-aaf6-50c125b24779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:04.711302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8068812-c3ac-4bad-aaf6-50c125b24779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:04.711586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8068812-c3ac-4bad-aaf6-50c125b24779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:04.711871 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8068812-c3ac-4bad-aaf6-50c125b24779 admin admin] PATH_INFO: `/v3/projects/c2d14886dbbb4cbaacd023a46686c4a5/users/434f772965724793bf76da0164fa88e8/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:04.728052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8068812-c3ac-4bad-aaf6-50c125b24779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d14886dbbb4cbaacd023a46686c4a5, user_id=434f772965724793bf76da0164fa88e8, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:04.731331 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8068812-c3ac-4bad-aaf6-50c125b24779 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:04.745907 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1237/2491] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:04 2022] PUT /identity/v3/projects/c2d14886dbbb4cbaacd023a46686c4a5/users/434f772965724793bf76da0164fa88e8/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:04.757180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600afd7a-0765-414a-8c63-2f733ba5af39 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:04.757513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600afd7a-0765-414a-8c63-2f733ba5af39 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:04.757769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600afd7a-0765-414a-8c63-2f733ba5af39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:04.794009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-600afd7a-0765-414a-8c63-2f733ba5af39 None None] MFA Rules not processed for user `434f772965724793bf76da0164fa88e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:04.795221 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-600afd7a-0765-414a-8c63-2f733ba5af39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:04.829061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:04.873457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1255/2492] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:04.878080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8068812-c3ac-4bad-aaf6-50c125b24779 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:04.949971 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8068812-c3ac-4bad-aaf6-50c125b24779 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:04.955019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172712fc-0cfd-4693-b59c-dc0438c4c8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:04.955644 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172712fc-0cfd-4693-b59c-dc0438c4c8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:04.955927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172712fc-0cfd-4693-b59c-dc0438c4c8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:04.956203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172712fc-0cfd-4693-b59c-dc0438c4c8d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:04.956913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172712fc-0cfd-4693-b59c-dc0438c4c8d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:04.958837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172712fc-0cfd-4693-b59c-dc0438c4c8d3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:04.988286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1238/2493] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:04 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:04.992225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600afd7a-0765-414a-8c63-2f733ba5af39 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:05.003663 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600afd7a-0765-414a-8c63-2f733ba5af39 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:05.066121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca769ebb-ef89-427d-a6a4-ac490e601353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:05.066731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769ebb-ef89-427d-a6a4-ac490e601353 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:05.067009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769ebb-ef89-427d-a6a4-ac490e601353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:05.067278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769ebb-ef89-427d-a6a4-ac490e601353 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:05.068266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca769ebb-ef89-427d-a6a4-ac490e601353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:05.071294 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca769ebb-ef89-427d-a6a4-ac490e601353 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:05.094158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1256/2494] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:04 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:05.098160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172712fc-0cfd-4693-b59c-dc0438c4c8d3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:05.110756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172712fc-0cfd-4693-b59c-dc0438c4c8d3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:05.115545 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc38a5-3814-41e8-b2df-ee1b4f0651a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:05.116155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc38a5-3814-41e8-b2df-ee1b4f0651a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:05.116436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc38a5-3814-41e8-b2df-ee1b4f0651a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:05.116765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc38a5-3814-41e8-b2df-ee1b4f0651a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:05.117409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc38a5-3814-41e8-b2df-ee1b4f0651a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:05.119568 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc38a5-3814-41e8-b2df-ee1b4f0651a2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:05.131724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1239/2495] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:05.137233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca769ebb-ef89-427d-a6a4-ac490e601353 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:05.147964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca769ebb-ef89-427d-a6a4-ac490e601353 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:05.153041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32321c-9b63-411b-8897-3fadb0abcd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:05.153699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32321c-9b63-411b-8897-3fadb0abcd5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:05.153981 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32321c-9b63-411b-8897-3fadb0abcd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:05.154261 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32321c-9b63-411b-8897-3fadb0abcd5d admin admin] PATH_INFO: `/v3/projects/caf5d7e6cde74a08912c3daf599422e3/users/b26fc6f4deef43168c557288b137bb93/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:05.171837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32321c-9b63-411b-8897-3fadb0abcd5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf5d7e6cde74a08912c3daf599422e3, user_id=b26fc6f4deef43168c557288b137bb93, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:05.172580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32321c-9b63-411b-8897-3fadb0abcd5d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:05.189622 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1257/2496] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:05 2022] PUT /identity/v3/projects/caf5d7e6cde74a08912c3daf599422e3/users/b26fc6f4deef43168c557288b137bb93/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:05.193507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc38a5-3814-41e8-b2df-ee1b4f0651a2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:05.205189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc38a5-3814-41e8-b2df-ee1b4f0651a2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:05.266178 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebabb0-2ede-4ee9-ae4f-681e5d0164d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:05.266794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebabb0-2ede-4ee9-ae4f-681e5d0164d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:05.267080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebabb0-2ede-4ee9-ae4f-681e5d0164d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:05.267359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebabb0-2ede-4ee9-ae4f-681e5d0164d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:05.268012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebabb0-2ede-4ee9-ae4f-681e5d0164d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:05.269656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebabb0-2ede-4ee9-ae4f-681e5d0164d7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:05.281516 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1240/2497] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:05.286092 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32321c-9b63-411b-8897-3fadb0abcd5d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:05.297790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32321c-9b63-411b-8897-3fadb0abcd5d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:05.302392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e119a-9442-4fd9-90aa-da3724dcd55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:05.303060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e119a-9442-4fd9-90aa-da3724dcd55a admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:05.303306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e119a-9442-4fd9-90aa-da3724dcd55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:05.303575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e119a-9442-4fd9-90aa-da3724dcd55a admin admin] PATH_INFO: `/v3/projects/caf5d7e6cde74a08912c3daf599422e3/users/b26fc6f4deef43168c557288b137bb93/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:05.305513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e119a-9442-4fd9-90aa-da3724dcd55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf5d7e6cde74a08912c3daf599422e3, user_id=b26fc6f4deef43168c557288b137bb93, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:05.308175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e119a-9442-4fd9-90aa-da3724dcd55a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:05.318731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1258/2498] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:05 2022] PUT /identity/v3/projects/caf5d7e6cde74a08912c3daf599422e3/users/b26fc6f4deef43168c557288b137bb93/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:05.330627 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28e16b-10e4-4ef3-b2f4-0ff3fd5a143d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:05.330913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28e16b-10e4-4ef3-b2f4-0ff3fd5a143d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:05.331193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28e16b-10e4-4ef3-b2f4-0ff3fd5a143d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:05.360252 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6f28e16b-10e4-4ef3-b2f4-0ff3fd5a143d None None] MFA Rules not processed for user `b26fc6f4deef43168c557288b137bb93`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:05.361754 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6f28e16b-10e4-4ef3-b2f4-0ff3fd5a143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:05.395885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:05.434825 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1241/2499] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:05.446839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e119a-9442-4fd9-90aa-da3724dcd55a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:05.449371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-218e119a-9442-4fd9-90aa-da3724dcd55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:05.514326 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e119a-9442-4fd9-90aa-da3724dcd55a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:05.518427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb0339-d70e-4bd5-8b0e-40a8e5f70f97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:05.519135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb0339-d70e-4bd5-8b0e-40a8e5f70f97 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:05.519462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb0339-d70e-4bd5-8b0e-40a8e5f70f97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:05.519773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb0339-d70e-4bd5-8b0e-40a8e5f70f97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:05.532210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb0339-d70e-4bd5-8b0e-40a8e5f70f97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:05.533775 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb0339-d70e-4bd5-8b0e-40a8e5f70f97 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:05.545244 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1259/2500] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:05.977028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28e16b-10e4-4ef3-b2f4-0ff3fd5a143d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:06.047732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28e16b-10e4-4ef3-b2f4-0ff3fd5a143d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:06.054560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:06.055394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:06.055776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:06.056146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] PATH_INFO: `/v3/users/434f772965724793bf76da0164fa88e8` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:06.057654 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=434f772965724793bf76da0164fa88e8)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:06.060394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:06.114330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434f772965724793bf76da0164fa88e8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.114987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434f772965724793bf76da0164fa88e8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.122778 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434f772965724793bf76da0164fa88e8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.131051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434f772965724793bf76da0164fa88e8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.131780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434f772965724793bf76da0164fa88e8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.150202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1242/2501] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:54:05 2022] DELETE /identity/v3/users/434f772965724793bf76da0164fa88e8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:06.155755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07d52e-2f12-4d52-8bdc-00d51103740e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:06.156045 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07d52e-2f12-4d52-8bdc-00d51103740e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:06.156313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07d52e-2f12-4d52-8bdc-00d51103740e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:06.175088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1c07d52e-2f12-4d52-8bdc-00d51103740e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:06.176285 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1c07d52e-2f12-4d52-8bdc-00d51103740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:06.195952 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:06.251727 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1260/2502] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:06.265674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:06.267305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:06.326959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:06.331003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ba492-be81-4095-9d96-41e0846dcf66 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:06.343232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbcd93-9712-40c5-9fef-6c7f1186089a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:06.343904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbcd93-9712-40c5-9fef-6c7f1186089a service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:06.344287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbcd93-9712-40c5-9fef-6c7f1186089a service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:06.344458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbcd93-9712-40c5-9fef-6c7f1186089a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:06.355247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbcd93-9712-40c5-9fef-6c7f1186089a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:06.356956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbcd93-9712-40c5-9fef-6c7f1186089a service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:06.366698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1243/2503] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:06.661530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07d52e-2f12-4d52-8bdc-00d51103740e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:06.663615 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1c07d52e-2f12-4d52-8bdc-00d51103740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:06.686556 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07d52e-2f12-4d52-8bdc-00d51103740e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:06.689666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:06.690354 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:06.690876 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:06.691174 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] PATH_INFO: `/v3/projects/c2d14886dbbb4cbaacd023a46686c4a5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:06.692363 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2d14886dbbb4cbaacd023a46686c4a5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:06.694660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:06.749884 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2d14886dbbb4cbaacd023a46686c4a5', 'request_id': 'req-5212854a-e1e8-45a5-9107-1b49151f376e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.756349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2d14886dbbb4cbaacd023a46686c4a5', 'request_id': 'req-5212854a-e1e8-45a5-9107-1b49151f376e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.763870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2d14886dbbb4cbaacd023a46686c4a5', 'request_id': 'req-5212854a-e1e8-45a5-9107-1b49151f376e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.779149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] The token cache is being invalidate because project c2d14886dbbb4cbaacd023a46686c4a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:06.779492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.780096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.781624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1261/2504] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:06 2022] DELETE /identity/v3/projects/c2d14886dbbb4cbaacd023a46686c4a5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:06.786732 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbcd93-9712-40c5-9fef-6c7f1186089a service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:06.798385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbcd93-9712-40c5-9fef-6c7f1186089a service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:06.858478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4524a85-2272-40f1-9c7d-c72f39b94c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:06.859113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524a85-2272-40f1-9c7d-c72f39b94c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:06.859414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524a85-2272-40f1-9c7d-c72f39b94c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:06.859792 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524a85-2272-40f1-9c7d-c72f39b94c0d admin admin] PATH_INFO: `/v3/users/b26fc6f4deef43168c557288b137bb93` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:06.861108 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4524a85-2272-40f1-9c7d-c72f39b94c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b26fc6f4deef43168c557288b137bb93)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:06.863217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4524a85-2272-40f1-9c7d-c72f39b94c0d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:06.915699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a4524a85-2272-40f1-9c7d-c72f39b94c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b26fc6f4deef43168c557288b137bb93'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.917698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a4524a85-2272-40f1-9c7d-c72f39b94c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b26fc6f4deef43168c557288b137bb93'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.925246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a4524a85-2272-40f1-9c7d-c72f39b94c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b26fc6f4deef43168c557288b137bb93'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.933918 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a4524a85-2272-40f1-9c7d-c72f39b94c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b26fc6f4deef43168c557288b137bb93'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.934559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a4524a85-2272-40f1-9c7d-c72f39b94c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b26fc6f4deef43168c557288b137bb93'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:06.955782 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1244/2505] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:54:06 2022] DELETE /identity/v3/users/b26fc6f4deef43168c557288b137bb93 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:07.304130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:07.305836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:07.325990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212854a-e1e8-45a5-9107-1b49151f376e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:07.389253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:07.389894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:07.390175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:07.390447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] PATH_INFO: `/v3/projects/caf5d7e6cde74a08912c3daf599422e3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:07.391562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] RBAC: Authorizing `identity:delete_project(project_id=caf5d7e6cde74a08912c3daf599422e3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:07.393520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:07.447393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:07.449009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caf5d7e6cde74a08912c3daf599422e3', 'request_id': 'req-aaad591c-7078-4a74-a01a-e77877fae94c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:07.459537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caf5d7e6cde74a08912c3daf599422e3', 'request_id': 'req-aaad591c-7078-4a74-a01a-e77877fae94c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:07.468819 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caf5d7e6cde74a08912c3daf599422e3', 'request_id': 'req-aaad591c-7078-4a74-a01a-e77877fae94c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:07.485745 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] The token cache is being invalidate because project caf5d7e6cde74a08912c3daf599422e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:07.486128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:07.486932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aaad591c-7078-4a74-a01a-e77877fae94c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:07.489242 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1262/2506] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:07 2022] DELETE /identity/v3/projects/caf5d7e6cde74a08912c3daf599422e3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:54:07.497260 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5753c-46c9-4d5d-b8f2-730a552490df None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:07.497745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5753c-46c9-4d5d-b8f2-730a552490df None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:07.498216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5753c-46c9-4d5d-b8f2-730a552490df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:07.525036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-abb5753c-46c9-4d5d-b8f2-730a552490df None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:07.526870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-abb5753c-46c9-4d5d-b8f2-730a552490df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:07.596791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1245/2507] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:07.605987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1e8ac-e383-4ac1-bf0c-947b809e9075 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:07.606353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1e8ac-e383-4ac1-bf0c-947b809e9075 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:07.606633 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1e8ac-e383-4ac1-bf0c-947b809e9075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:07.627100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cfd1e8ac-e383-4ac1-bf0c-947b809e9075 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:07.628297 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cfd1e8ac-e383-4ac1-bf0c-947b809e9075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:07.637395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1263/2508] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:07.641744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5753c-46c9-4d5d-b8f2-730a552490df None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:07.652270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5753c-46c9-4d5d-b8f2-730a552490df None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:07.657450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dc44d-4258-4d2c-a0fa-2b12d60f06b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:07.658176 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dc44d-4258-4d2c-a0fa-2b12d60f06b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:07.658494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dc44d-4258-4d2c-a0fa-2b12d60f06b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:07.658776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dc44d-4258-4d2c-a0fa-2b12d60f06b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:07.659442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dc44d-4258-4d2c-a0fa-2b12d60f06b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:54:07.659757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dc44d-4258-4d2c-a0fa-2b12d60f06b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:07.661430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dc44d-4258-4d2c-a0fa-2b12d60f06b2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:07.669403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:07.675376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1246/2509] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:54:07.680162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1e8ac-e383-4ac1-bf0c-947b809e9075 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:07.692550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1e8ac-e383-4ac1-bf0c-947b809e9075 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:07.699249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfb010-3ad9-4f86-8aff-535951498c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:07.699866 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb010-3ad9-4f86-8aff-535951498c67 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:07.700150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb010-3ad9-4f86-8aff-535951498c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:07.700427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb010-3ad9-4f86-8aff-535951498c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:07.701135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfb010-3ad9-4f86-8aff-535951498c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:07.703100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfb010-3ad9-4f86-8aff-535951498c67 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:07.733584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1264/2510] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:07 2022] POST /identity/v3/projects => generated 401 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:07.737966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dc44d-4258-4d2c-a0fa-2b12d60f06b2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:07.748316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dc44d-4258-4d2c-a0fa-2b12d60f06b2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:07.810677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f9a55-6bc0-4e2a-9ad2-8825baa37984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:07.811372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f9a55-6bc0-4e2a-9ad2-8825baa37984 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:07.811570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f9a55-6bc0-4e2a-9ad2-8825baa37984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:07.811853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f9a55-6bc0-4e2a-9ad2-8825baa37984 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:07.812517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f9a55-6bc0-4e2a-9ad2-8825baa37984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:07.814630 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f9a55-6bc0-4e2a-9ad2-8825baa37984 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:07.840420 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1247/2511] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:07 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:07.844629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfb010-3ad9-4f86-8aff-535951498c67 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:07.856867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfb010-3ad9-4f86-8aff-535951498c67 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:07.862698 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d0c11-0d19-4598-9c66-5d9a3dbccaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:07.863502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d0c11-0d19-4598-9c66-5d9a3dbccaaa admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:07.863889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d0c11-0d19-4598-9c66-5d9a3dbccaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:07.864235 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d0c11-0d19-4598-9c66-5d9a3dbccaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:07.865044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d0c11-0d19-4598-9c66-5d9a3dbccaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:07.867043 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d0c11-0d19-4598-9c66-5d9a3dbccaaa admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:07.879048 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1265/2512] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:54:07.883721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f9a55-6bc0-4e2a-9ad2-8825baa37984 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:07.900996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f9a55-6bc0-4e2a-9ad2-8825baa37984 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:07.907435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efb930-31af-4f22-9b7e-89e0e8b59197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:07.908077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efb930-31af-4f22-9b7e-89e0e8b59197 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:07.908360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efb930-31af-4f22-9b7e-89e0e8b59197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:07.908667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efb930-31af-4f22-9b7e-89e0e8b59197 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:07.909343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efb930-31af-4f22-9b7e-89e0e8b59197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:07.910990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efb930-31af-4f22-9b7e-89e0e8b59197 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:07.924096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1248/2513] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:07.928482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d0c11-0d19-4598-9c66-5d9a3dbccaaa admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:07.940030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d0c11-0d19-4598-9c66-5d9a3dbccaaa admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:07.945208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70ad79-0cff-4534-8085-531264597f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:07.945860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70ad79-0cff-4534-8085-531264597f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:07.946141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70ad79-0cff-4534-8085-531264597f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:07.946412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70ad79-0cff-4534-8085-531264597f45 admin admin] PATH_INFO: `/v3/projects/0d141fa68e4d4606ae28c237e3f1eb50/users/2677107b4d854082976b1b9b0d75181a/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:07.962848 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70ad79-0cff-4534-8085-531264597f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d141fa68e4d4606ae28c237e3f1eb50, user_id=2677107b4d854082976b1b9b0d75181a, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:07.966014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70ad79-0cff-4534-8085-531264597f45 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:07.979778 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1266/2514] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:07 2022] PUT /identity/v3/projects/0d141fa68e4d4606ae28c237e3f1eb50/users/2677107b4d854082976b1b9b0d75181a/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:07.991843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9d872-5b29-476e-80bb-afcbac100248 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:07.992138 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9d872-5b29-476e-80bb-afcbac100248 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:07.992569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9d872-5b29-476e-80bb-afcbac100248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:08.023797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6df9d872-5b29-476e-80bb-afcbac100248 None None] MFA Rules not processed for user `2677107b4d854082976b1b9b0d75181a`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:08.025197 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6df9d872-5b29-476e-80bb-afcbac100248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:08.113701 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1249/2515] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:07 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:08.123994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70ad79-0cff-4534-8085-531264597f45 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:08.189040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70ad79-0cff-4534-8085-531264597f45 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:08.194144 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc60afe-13aa-4a24-a0e8-f9e29b18ad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:08.194762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc60afe-13aa-4a24-a0e8-f9e29b18ad97 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:08.195044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc60afe-13aa-4a24-a0e8-f9e29b18ad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:08.195321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc60afe-13aa-4a24-a0e8-f9e29b18ad97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:08.195954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc60afe-13aa-4a24-a0e8-f9e29b18ad97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:08.198667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc60afe-13aa-4a24-a0e8-f9e29b18ad97 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:08.225097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1267/2516] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:08 2022] POST /identity/v3/projects => generated 401 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:08.229796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9d872-5b29-476e-80bb-afcbac100248 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:08.240723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9d872-5b29-476e-80bb-afcbac100248 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:08.302794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ec22a-27ad-47a4-869a-ed48fb7bc902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:08.303429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ec22a-27ad-47a4-869a-ed48fb7bc902 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:08.303714 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ec22a-27ad-47a4-869a-ed48fb7bc902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:08.303993 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ec22a-27ad-47a4-869a-ed48fb7bc902 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:08.304688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ec22a-27ad-47a4-869a-ed48fb7bc902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:08.307387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ec22a-27ad-47a4-869a-ed48fb7bc902 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:08.319344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:08.331716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1250/2517] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:08 2022] POST /identity/v3/users => generated 307 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:08.335557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc60afe-13aa-4a24-a0e8-f9e29b18ad97 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:08.347225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc60afe-13aa-4a24-a0e8-f9e29b18ad97 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:08.351995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98081532-4789-41d9-b570-8513d973c9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:08.352761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98081532-4789-41d9-b570-8513d973c9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:08.353007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98081532-4789-41d9-b570-8513d973c9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:08.353803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98081532-4789-41d9-b570-8513d973c9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:08.354725 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98081532-4789-41d9-b570-8513d973c9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:08.356324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98081532-4789-41d9-b570-8513d973c9e4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:08.367173 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1268/2518] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:08.372306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ec22a-27ad-47a4-869a-ed48fb7bc902 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:08.382833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ec22a-27ad-47a4-869a-ed48fb7bc902 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:08.388256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827ed62-ebfb-4067-b6ce-033dd432fd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:08.388954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827ed62-ebfb-4067-b6ce-033dd432fd0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:08.389269 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827ed62-ebfb-4067-b6ce-033dd432fd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:08.389548 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827ed62-ebfb-4067-b6ce-033dd432fd0a admin admin] PATH_INFO: `/v3/projects/9f07bdf3ff2647828af7eeab66dc6184/users/e71acbba00ee4eac8fa3abb7a30e4beb/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:08.404921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827ed62-ebfb-4067-b6ce-033dd432fd0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f07bdf3ff2647828af7eeab66dc6184, user_id=e71acbba00ee4eac8fa3abb7a30e4beb, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:08.407606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827ed62-ebfb-4067-b6ce-033dd432fd0a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:08.421109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1251/2519] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:08 2022] PUT /identity/v3/projects/9f07bdf3ff2647828af7eeab66dc6184/users/e71acbba00ee4eac8fa3abb7a30e4beb/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:08.425048 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98081532-4789-41d9-b570-8513d973c9e4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:08.436022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98081532-4789-41d9-b570-8513d973c9e4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:08.491178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:08.507123 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6eb51-853c-47df-aadb-50299f5f06e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:08.507734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6eb51-853c-47df-aadb-50299f5f06e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:08.508016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6eb51-853c-47df-aadb-50299f5f06e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:08.508291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6eb51-853c-47df-aadb-50299f5f06e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:08.508964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6eb51-853c-47df-aadb-50299f5f06e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:08.513987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6eb51-853c-47df-aadb-50299f5f06e1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:08.528576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1269/2520] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:08.532408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827ed62-ebfb-4067-b6ce-033dd432fd0a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:08.546276 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827ed62-ebfb-4067-b6ce-033dd432fd0a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:08.551280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04a130-a480-4811-8e3a-6d83b3669c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:08.551969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04a130-a480-4811-8e3a-6d83b3669c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:08.552286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04a130-a480-4811-8e3a-6d83b3669c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:08.552594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04a130-a480-4811-8e3a-6d83b3669c37 admin admin] PATH_INFO: `/v3/projects/9f07bdf3ff2647828af7eeab66dc6184/users/e71acbba00ee4eac8fa3abb7a30e4beb/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:08.554501 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04a130-a480-4811-8e3a-6d83b3669c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f07bdf3ff2647828af7eeab66dc6184, user_id=e71acbba00ee4eac8fa3abb7a30e4beb, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:08.558097 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04a130-a480-4811-8e3a-6d83b3669c37 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:08.570226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1252/2521] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:08 2022] PUT /identity/v3/projects/9f07bdf3ff2647828af7eeab66dc6184/users/e71acbba00ee4eac8fa3abb7a30e4beb/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:08.581540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1963c-c6fd-46e3-a9de-5a0d1756bb5c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:08.582222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1963c-c6fd-46e3-a9de-5a0d1756bb5c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:08.582482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1963c-c6fd-46e3-a9de-5a0d1756bb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:08.611949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3dd1963c-c6fd-46e3-a9de-5a0d1756bb5c None None] MFA Rules not processed for user `e71acbba00ee4eac8fa3abb7a30e4beb`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:08.613200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3dd1963c-c6fd-46e3-a9de-5a0d1756bb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:08.691873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1270/2522] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:08 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:08.702976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04a130-a480-4811-8e3a-6d83b3669c37 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:08.704727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-be04a130-a480-4811-8e3a-6d83b3669c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:08.768891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04a130-a480-4811-8e3a-6d83b3669c37 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:08.772752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1da48-3dc3-4205-9e67-53a9f7bff7db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:08.773371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1da48-3dc3-4205-9e67-53a9f7bff7db service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:08.773658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1da48-3dc3-4205-9e67-53a9f7bff7db service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:08.773933 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1da48-3dc3-4205-9e67-53a9f7bff7db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:08.783806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1da48-3dc3-4205-9e67-53a9f7bff7db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:08.785309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1da48-3dc3-4205-9e67-53a9f7bff7db service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:08.795462 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1253/2523] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:08 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:08.895953 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1963c-c6fd-46e3-a9de-5a0d1756bb5c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:08.906769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1963c-c6fd-46e3-a9de-5a0d1756bb5c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:08.910485 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa50df-e831-4e4c-a6ad-d7c35d953618 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:08.911101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa50df-e831-4e4c-a6ad-d7c35d953618 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:08.911391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa50df-e831-4e4c-a6ad-d7c35d953618 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:08.911668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa50df-e831-4e4c-a6ad-d7c35d953618 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:08.979337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa50df-e831-4e4c-a6ad-d7c35d953618 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:08.980717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa50df-e831-4e4c-a6ad-d7c35d953618 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:08.989918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1271/2524] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:08 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:11.781613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1da48-3dc3-4205-9e67-53a9f7bff7db service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:11.783071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a6d1da48-3dc3-4205-9e67-53a9f7bff7db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:11.843034 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1da48-3dc3-4205-9e67-53a9f7bff7db service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:11.846920 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbe4eb-447e-4907-88dc-f5f44ae70bbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:11.847615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbe4eb-447e-4907-88dc-f5f44ae70bbe service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:11.848084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbe4eb-447e-4907-88dc-f5f44ae70bbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:11.848235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbe4eb-447e-4907-88dc-f5f44ae70bbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:11.850060 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-76dbe4eb-447e-4907-88dc-f5f44ae70bbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:11.862038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbe4eb-447e-4907-88dc-f5f44ae70bbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:11.863896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbe4eb-447e-4907-88dc-f5f44ae70bbe service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:11.873547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1254/2525] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:13.152250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa50df-e831-4e4c-a6ad-d7c35d953618 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:13.219531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa50df-e831-4e4c-a6ad-d7c35d953618 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:13.224073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:13.224720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:13.224996 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:13.225262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] PATH_INFO: `/v3/users/2677107b4d854082976b1b9b0d75181a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:13.226356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] RBAC: Authorizing `identity:delete_user(user_id=2677107b4d854082976b1b9b0d75181a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:13.228401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:13.233933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:13.281390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2677107b4d854082976b1b9b0d75181a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:13.290260 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2677107b4d854082976b1b9b0d75181a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:13.291246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2677107b4d854082976b1b9b0d75181a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:13.292049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2677107b4d854082976b1b9b0d75181a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:13.299946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2677107b4d854082976b1b9b0d75181a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:13.318793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1272/2526] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:54:13 2022] DELETE /identity/v3/users/2677107b4d854082976b1b9b0d75181a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:54:13.323946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34501beb-96d2-4736-a26d-ebbe6b3d7404 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:13.324460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34501beb-96d2-4736-a26d-ebbe6b3d7404 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:13.324656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34501beb-96d2-4736-a26d-ebbe6b3d7404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:13.344897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-34501beb-96d2-4736-a26d-ebbe6b3d7404 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:13.346677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-34501beb-96d2-4736-a26d-ebbe6b3d7404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:13.421999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1255/2527] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:13.433595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:13.435126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:13.491645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a3272-6b34-4993-be55-7e1e4ee352da admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:13.495906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194382e-1c0e-49a2-ba49-aaab0cf2a1bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:13.496528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194382e-1c0e-49a2-ba49-aaab0cf2a1bb service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:13.496809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194382e-1c0e-49a2-ba49-aaab0cf2a1bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:13.497074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194382e-1c0e-49a2-ba49-aaab0cf2a1bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:13.506448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194382e-1c0e-49a2-ba49-aaab0cf2a1bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:13.507876 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194382e-1c0e-49a2-ba49-aaab0cf2a1bb service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:13.518120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1273/2528] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:13.825020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34501beb-96d2-4736-a26d-ebbe6b3d7404 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:13.826532 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-34501beb-96d2-4736-a26d-ebbe6b3d7404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:13.836827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:13.842378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34501beb-96d2-4736-a26d-ebbe6b3d7404 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:13.850358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:13.851152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:13.851755 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:13.852180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] PATH_INFO: `/v3/projects/0d141fa68e4d4606ae28c237e3f1eb50` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:13.854140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d141fa68e4d4606ae28c237e3f1eb50)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:13.856659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:13.908595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d141fa68e4d4606ae28c237e3f1eb50', 'request_id': 'req-8e6dd04e-c1ba-417b-9092-2498d7623323'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:13.914448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d141fa68e4d4606ae28c237e3f1eb50', 'request_id': 'req-8e6dd04e-c1ba-417b-9092-2498d7623323'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:13.921651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d141fa68e4d4606ae28c237e3f1eb50', 'request_id': 'req-8e6dd04e-c1ba-417b-9092-2498d7623323'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:13.935469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] The token cache is being invalidate because project 0d141fa68e4d4606ae28c237e3f1eb50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:13.935766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:13.936403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:13.937985 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1256/2529] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:13 2022] DELETE /identity/v3/projects/0d141fa68e4d4606ae28c237e3f1eb50 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:13.941987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194382e-1c0e-49a2-ba49-aaab0cf2a1bb service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:13.951856 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194382e-1c0e-49a2-ba49-aaab0cf2a1bb service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:14.015946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f027c6c-b630-4ec3-ae64-12f1008b9fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:14.016608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f027c6c-b630-4ec3-ae64-12f1008b9fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:14.016894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f027c6c-b630-4ec3-ae64-12f1008b9fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:14.017164 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f027c6c-b630-4ec3-ae64-12f1008b9fde admin admin] PATH_INFO: `/v3/users/e71acbba00ee4eac8fa3abb7a30e4beb` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:14.018265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f027c6c-b630-4ec3-ae64-12f1008b9fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71acbba00ee4eac8fa3abb7a30e4beb)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:14.020163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f027c6c-b630-4ec3-ae64-12f1008b9fde admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:14.052730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:14.070643 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f027c6c-b630-4ec3-ae64-12f1008b9fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71acbba00ee4eac8fa3abb7a30e4beb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:14.078985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f027c6c-b630-4ec3-ae64-12f1008b9fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71acbba00ee4eac8fa3abb7a30e4beb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:14.079651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f027c6c-b630-4ec3-ae64-12f1008b9fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71acbba00ee4eac8fa3abb7a30e4beb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:14.080253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f027c6c-b630-4ec3-ae64-12f1008b9fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71acbba00ee4eac8fa3abb7a30e4beb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:14.088551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f027c6c-b630-4ec3-ae64-12f1008b9fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71acbba00ee4eac8fa3abb7a30e4beb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:14.107790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1274/2530] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:54:13 2022] DELETE /identity/v3/users/e71acbba00ee4eac8fa3abb7a30e4beb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:14.403856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:14.405434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:14.419086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dd04e-c1ba-417b-9092-2498d7623323 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:14.501236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:14.501886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:14.502175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:14.502454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] PATH_INFO: `/v3/projects/9f07bdf3ff2647828af7eeab66dc6184` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:14.503596 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f07bdf3ff2647828af7eeab66dc6184)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:14.505569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:14.553895 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f07bdf3ff2647828af7eeab66dc6184', 'request_id': 'req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:14.560088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f07bdf3ff2647828af7eeab66dc6184', 'request_id': 'req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:14.567134 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f07bdf3ff2647828af7eeab66dc6184', 'request_id': 'req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:14.581469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] The token cache is being invalidate because project 9f07bdf3ff2647828af7eeab66dc6184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:14.581785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:14.582387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d7f63ba7-1c55-45f4-9732-6f7bf274e11a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:14.583906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1257/2531] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:14 2022] DELETE /identity/v3/projects/9f07bdf3ff2647828af7eeab66dc6184 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:54:14.589634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca097d-8ed9-4b12-be1d-ace57d2e487e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:14.590153 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca097d-8ed9-4b12-be1d-ace57d2e487e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:14.590348 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca097d-8ed9-4b12-be1d-ace57d2e487e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:14.611101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-23ca097d-8ed9-4b12-be1d-ace57d2e487e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:14.612321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-23ca097d-8ed9-4b12-be1d-ace57d2e487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:14.673384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1275/2532] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:14.684161 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97229afe-1d56-4d0f-9561-c64c7fbc0b44 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:14.684480 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97229afe-1d56-4d0f-9561-c64c7fbc0b44 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:14.684818 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97229afe-1d56-4d0f-9561-c64c7fbc0b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:14.705577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-97229afe-1d56-4d0f-9561-c64c7fbc0b44 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:14.706800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-97229afe-1d56-4d0f-9561-c64c7fbc0b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:14.715520 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1258/2533] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:14.719971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca097d-8ed9-4b12-be1d-ace57d2e487e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:14.730287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca097d-8ed9-4b12-be1d-ace57d2e487e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:14.735433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dadbf-1115-4fcf-a525-71fd7a1260af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:14.736036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dadbf-1115-4fcf-a525-71fd7a1260af admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:14.736317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dadbf-1115-4fcf-a525-71fd7a1260af admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:14.736627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dadbf-1115-4fcf-a525-71fd7a1260af admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:14.737276 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dadbf-1115-4fcf-a525-71fd7a1260af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:54:14.737577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dadbf-1115-4fcf-a525-71fd7a1260af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:14.739165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dadbf-1115-4fcf-a525-71fd7a1260af admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:14.751480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1276/2534] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:54:14.755964 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97229afe-1d56-4d0f-9561-c64c7fbc0b44 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:14.766126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97229afe-1d56-4d0f-9561-c64c7fbc0b44 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:14.770980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a1663-642c-4e8e-9ddb-feffe894ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:14.771592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a1663-642c-4e8e-9ddb-feffe894ccf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:14.771879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a1663-642c-4e8e-9ddb-feffe894ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:14.772147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a1663-642c-4e8e-9ddb-feffe894ccf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:14.772829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a1663-642c-4e8e-9ddb-feffe894ccf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:14.775300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a1663-642c-4e8e-9ddb-feffe894ccf8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:14.785435 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:14.805489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1259/2535] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:14 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:14.810175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dadbf-1115-4fcf-a525-71fd7a1260af admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:14.820470 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dadbf-1115-4fcf-a525-71fd7a1260af admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:14.879839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b129fb1-a090-4727-97b1-e13c10b78b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:14.880445 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b129fb1-a090-4727-97b1-e13c10b78b21 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:14.880761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b129fb1-a090-4727-97b1-e13c10b78b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:14.881039 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b129fb1-a090-4727-97b1-e13c10b78b21 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:14.881681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b129fb1-a090-4727-97b1-e13c10b78b21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:14.883613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b129fb1-a090-4727-97b1-e13c10b78b21 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:14.905380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1277/2536] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:14 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:14.909630 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a1663-642c-4e8e-9ddb-feffe894ccf8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:14.920799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a1663-642c-4e8e-9ddb-feffe894ccf8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:14.925511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13783c02-0ab6-4c3c-9454-1c648f458af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:14.926120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13783c02-0ab6-4c3c-9454-1c648f458af0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:14.926402 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13783c02-0ab6-4c3c-9454-1c648f458af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:14.926675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13783c02-0ab6-4c3c-9454-1c648f458af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:14.927313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13783c02-0ab6-4c3c-9454-1c648f458af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:14.928961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13783c02-0ab6-4c3c-9454-1c648f458af0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:14.939657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1260/2537] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:14.943849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b129fb1-a090-4727-97b1-e13c10b78b21 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:14.954133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b129fb1-a090-4727-97b1-e13c10b78b21 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:14.959335 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4ede9-dd07-44b8-ad51-9650c06c68ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:14.959940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4ede9-dd07-44b8-ad51-9650c06c68ba admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:14.960238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4ede9-dd07-44b8-ad51-9650c06c68ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:14.960507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4ede9-dd07-44b8-ad51-9650c06c68ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:14.961158 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4ede9-dd07-44b8-ad51-9650c06c68ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:14.963651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4ede9-dd07-44b8-ad51-9650c06c68ba admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:14.974886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1278/2538] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:14.980416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13783c02-0ab6-4c3c-9454-1c648f458af0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:14.990757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13783c02-0ab6-4c3c-9454-1c648f458af0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:14.997053 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a4d5b-5e36-4f5e-b7d9-ff6207b0f609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:14.997717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4d5b-5e36-4f5e-b7d9-ff6207b0f609 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:14.998010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4d5b-5e36-4f5e-b7d9-ff6207b0f609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:14.998296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4d5b-5e36-4f5e-b7d9-ff6207b0f609 admin admin] PATH_INFO: `/v3/projects/77571cf4b2a24c9a9cdda407be5f09c6/users/fee0ff3d3b534d90b0c824fe50a2e2b1/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.014538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a4d5b-5e36-4f5e-b7d9-ff6207b0f609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77571cf4b2a24c9a9cdda407be5f09c6, user_id=fee0ff3d3b534d90b0c824fe50a2e2b1, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:15.017277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a4d5b-5e36-4f5e-b7d9-ff6207b0f609 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:15.030492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1261/2539] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:14 2022] PUT /identity/v3/projects/77571cf4b2a24c9a9cdda407be5f09c6/users/fee0ff3d3b534d90b0c824fe50a2e2b1/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:15.041678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bba0b-353f-4999-a7fe-c8391f2f5531 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.041966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bba0b-353f-4999-a7fe-c8391f2f5531 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.042240 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bba0b-353f-4999-a7fe-c8391f2f5531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.070734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:15.074899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-115bba0b-353f-4999-a7fe-c8391f2f5531 None None] MFA Rules not processed for user `fee0ff3d3b534d90b0c824fe50a2e2b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:15.077146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-115bba0b-353f-4999-a7fe-c8391f2f5531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:15.144379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1279/2540] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:15.149031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a4d5b-5e36-4f5e-b7d9-ff6207b0f609 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:15.221247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a4d5b-5e36-4f5e-b7d9-ff6207b0f609 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:15.225911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e182fac-115b-43b2-97f4-a28051b9e848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:15.226553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e182fac-115b-43b2-97f4-a28051b9e848 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.226839 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e182fac-115b-43b2-97f4-a28051b9e848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.227116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e182fac-115b-43b2-97f4-a28051b9e848 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.228316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e182fac-115b-43b2-97f4-a28051b9e848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:15.230682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e182fac-115b-43b2-97f4-a28051b9e848 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:15.259505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1262/2541] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:15 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:15.263794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115bba0b-353f-4999-a7fe-c8391f2f5531 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:15.275296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115bba0b-353f-4999-a7fe-c8391f2f5531 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:15.339011 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1287ad-a5d2-4054-91ea-990727a609e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:15.339627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1287ad-a5d2-4054-91ea-990727a609e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.339906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1287ad-a5d2-4054-91ea-990727a609e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.340175 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1287ad-a5d2-4054-91ea-990727a609e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.340888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1287ad-a5d2-4054-91ea-990727a609e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:15.342822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1287ad-a5d2-4054-91ea-990727a609e1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:15.375608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1280/2542] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:15 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:54:15.380014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e182fac-115b-43b2-97f4-a28051b9e848 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:15.392800 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e182fac-115b-43b2-97f4-a28051b9e848 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:15.397940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58077b2-b78b-479b-ac8e-1fe3202a8be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:15.398554 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58077b2-b78b-479b-ac8e-1fe3202a8be9 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.398841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58077b2-b78b-479b-ac8e-1fe3202a8be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.399124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58077b2-b78b-479b-ac8e-1fe3202a8be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.399849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58077b2-b78b-479b-ac8e-1fe3202a8be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:15.402091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58077b2-b78b-479b-ac8e-1fe3202a8be9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:15.413425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1263/2543] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:54:15.418378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1287ad-a5d2-4054-91ea-990727a609e1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:15.428395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1287ad-a5d2-4054-91ea-990727a609e1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:15.433968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c220e-7268-44b5-ac6c-dd0a3baf9710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:15.434564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c220e-7268-44b5-ac6c-dd0a3baf9710 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.434846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c220e-7268-44b5-ac6c-dd0a3baf9710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.435120 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c220e-7268-44b5-ac6c-dd0a3baf9710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.435757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c220e-7268-44b5-ac6c-dd0a3baf9710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:15.437378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c220e-7268-44b5-ac6c-dd0a3baf9710 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:15.447692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1281/2544] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:15.451886 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58077b2-b78b-479b-ac8e-1fe3202a8be9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:15.462020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58077b2-b78b-479b-ac8e-1fe3202a8be9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:15.467337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132cfd3-32cf-4efc-b485-b7b7e837514f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:15.467978 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132cfd3-32cf-4efc-b485-b7b7e837514f admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.468257 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132cfd3-32cf-4efc-b485-b7b7e837514f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.468559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132cfd3-32cf-4efc-b485-b7b7e837514f admin admin] PATH_INFO: `/v3/projects/cc565a17f3134974b95ccc93ea962efe/users/1af339c95de7433caf45f97dc1e2bf50/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.483410 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132cfd3-32cf-4efc-b485-b7b7e837514f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc565a17f3134974b95ccc93ea962efe, user_id=1af339c95de7433caf45f97dc1e2bf50, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:15.486283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132cfd3-32cf-4efc-b485-b7b7e837514f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:15.499949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1264/2545] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:15 2022] PUT /identity/v3/projects/cc565a17f3134974b95ccc93ea962efe/users/1af339c95de7433caf45f97dc1e2bf50/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:15.510670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e185fa-7625-4095-9cff-e897eac3b92c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.510963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e185fa-7625-4095-9cff-e897eac3b92c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.511311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e185fa-7625-4095-9cff-e897eac3b92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.544917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a6e185fa-7625-4095-9cff-e897eac3b92c None None] MFA Rules not processed for user `1af339c95de7433caf45f97dc1e2bf50`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:15.546126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a6e185fa-7625-4095-9cff-e897eac3b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:15.612084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1282/2546] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:15.616967 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132cfd3-32cf-4efc-b485-b7b7e837514f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:15.682597 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132cfd3-32cf-4efc-b485-b7b7e837514f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:15.687560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f069d4-6b45-4668-a703-b26fb5b2120c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:15.688202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f069d4-6b45-4668-a703-b26fb5b2120c admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.688512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f069d4-6b45-4668-a703-b26fb5b2120c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.688795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f069d4-6b45-4668-a703-b26fb5b2120c admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.689492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f069d4-6b45-4668-a703-b26fb5b2120c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:15.691417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f069d4-6b45-4668-a703-b26fb5b2120c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:15.702051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:15.720560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1265/2547] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:15 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:15.724702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e185fa-7625-4095-9cff-e897eac3b92c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:15.735983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e185fa-7625-4095-9cff-e897eac3b92c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:15.743946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:15.794438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e11f91-7e28-4ea4-bd34-31c847fe9530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:15.795036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e11f91-7e28-4ea4-bd34-31c847fe9530 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.795313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e11f91-7e28-4ea4-bd34-31c847fe9530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.795667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e11f91-7e28-4ea4-bd34-31c847fe9530 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.796220 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e11f91-7e28-4ea4-bd34-31c847fe9530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:15.798165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e11f91-7e28-4ea4-bd34-31c847fe9530 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:15.819009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1283/2548] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:15 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:15.824081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f069d4-6b45-4668-a703-b26fb5b2120c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:15.834227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f069d4-6b45-4668-a703-b26fb5b2120c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:15.839707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04dc2a8-78c7-4f04-bd1d-996517c25f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:15.840302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dc2a8-78c7-4f04-bd1d-996517c25f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.840616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dc2a8-78c7-4f04-bd1d-996517c25f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.840888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dc2a8-78c7-4f04-bd1d-996517c25f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.841522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04dc2a8-78c7-4f04-bd1d-996517c25f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:15.843107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04dc2a8-78c7-4f04-bd1d-996517c25f8b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:15.853167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1266/2549] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:15.857463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e11f91-7e28-4ea4-bd34-31c847fe9530 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:15.867380 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e11f91-7e28-4ea4-bd34-31c847fe9530 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:15.872193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf29ab9-2987-4b40-bacc-1903b16ddc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:15.873023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf29ab9-2987-4b40-bacc-1903b16ddc25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.873165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf29ab9-2987-4b40-bacc-1903b16ddc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.873493 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf29ab9-2987-4b40-bacc-1903b16ddc25 admin admin] PATH_INFO: `/v3/projects/75df1313e8a04bb4bfce4ae213acdabd/users/d496f365e44e4cc2a7e6575d58239d72/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.888739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf29ab9-2987-4b40-bacc-1903b16ddc25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75df1313e8a04bb4bfce4ae213acdabd, user_id=d496f365e44e4cc2a7e6575d58239d72, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:15.891370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf29ab9-2987-4b40-bacc-1903b16ddc25 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:15.904888 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1284/2550] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:15 2022] PUT /identity/v3/projects/75df1313e8a04bb4bfce4ae213acdabd/users/d496f365e44e4cc2a7e6575d58239d72/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:15.909026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04dc2a8-78c7-4f04-bd1d-996517c25f8b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:15.919694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04dc2a8-78c7-4f04-bd1d-996517c25f8b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:15.975363 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe2be1-d10a-4c5d-9333-d16f91a1d050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:15.975969 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe2be1-d10a-4c5d-9333-d16f91a1d050 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:15.976254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe2be1-d10a-4c5d-9333-d16f91a1d050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:15.976601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe2be1-d10a-4c5d-9333-d16f91a1d050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:15.977251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe2be1-d10a-4c5d-9333-d16f91a1d050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:15.978905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe2be1-d10a-4c5d-9333-d16f91a1d050 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:15.989255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1267/2551] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:15.994013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf29ab9-2987-4b40-bacc-1903b16ddc25 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:16.005671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf29ab9-2987-4b40-bacc-1903b16ddc25 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:16.011604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01145045-796b-42fe-8147-5ffa865ea059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:16.012253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01145045-796b-42fe-8147-5ffa865ea059 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:16.012573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01145045-796b-42fe-8147-5ffa865ea059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:16.012851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01145045-796b-42fe-8147-5ffa865ea059 admin admin] PATH_INFO: `/v3/projects/75df1313e8a04bb4bfce4ae213acdabd/users/d496f365e44e4cc2a7e6575d58239d72/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:16.014756 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01145045-796b-42fe-8147-5ffa865ea059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75df1313e8a04bb4bfce4ae213acdabd, user_id=d496f365e44e4cc2a7e6575d58239d72, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:16.017450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01145045-796b-42fe-8147-5ffa865ea059 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:16.028712 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1285/2552] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:15 2022] PUT /identity/v3/projects/75df1313e8a04bb4bfce4ae213acdabd/users/d496f365e44e4cc2a7e6575d58239d72/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:16.039772 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b528dc5-26a7-4592-a969-a2e3452f3c12 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:16.040063 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b528dc5-26a7-4592-a969-a2e3452f3c12 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:16.040379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b528dc5-26a7-4592-a969-a2e3452f3c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:16.069635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6b528dc5-26a7-4592-a969-a2e3452f3c12 None None] MFA Rules not processed for user `d496f365e44e4cc2a7e6575d58239d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:16.070866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6b528dc5-26a7-4592-a969-a2e3452f3c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:16.140380 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1268/2553] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:16.153313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01145045-796b-42fe-8147-5ffa865ea059 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:16.158403 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-01145045-796b-42fe-8147-5ffa865ea059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:16.217741 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01145045-796b-42fe-8147-5ffa865ea059 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:16.221350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c60c4-3036-4e24-aa33-0719d97e8b4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:16.221949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c60c4-3036-4e24-aa33-0719d97e8b4c service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:16.222227 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c60c4-3036-4e24-aa33-0719d97e8b4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:16.222501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c60c4-3036-4e24-aa33-0719d97e8b4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:16.279306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c60c4-3036-4e24-aa33-0719d97e8b4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:16.280424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c60c4-3036-4e24-aa33-0719d97e8b4c service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:16.289997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1286/2554] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:18.896962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b528dc5-26a7-4592-a969-a2e3452f3c12 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:18.898466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6b528dc5-26a7-4592-a969-a2e3452f3c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:18.911215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b528dc5-26a7-4592-a969-a2e3452f3c12 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:18.914879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee565b52-9866-4680-8d76-c0d8932c80ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:18.915990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee565b52-9866-4680-8d76-c0d8932c80ea service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:18.916729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee565b52-9866-4680-8d76-c0d8932c80ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:18.916729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee565b52-9866-4680-8d76-c0d8932c80ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:18.976091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee565b52-9866-4680-8d76-c0d8932c80ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:18.978360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee565b52-9866-4680-8d76-c0d8932c80ea service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:18.982252 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:18.991619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1269/2555] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:54:19.609069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c60c4-3036-4e24-aa33-0719d97e8b4c service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:19.619084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c60c4-3036-4e24-aa33-0719d97e8b4c service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:19.623104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1704cae-3349-4c61-a3f7-5ce83a59a675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:19.623730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1704cae-3349-4c61-a3f7-5ce83a59a675 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:19.624009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1704cae-3349-4c61-a3f7-5ce83a59a675 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:19.624285 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1704cae-3349-4c61-a3f7-5ce83a59a675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:19.634063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1704cae-3349-4c61-a3f7-5ce83a59a675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:19.635683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1704cae-3349-4c61-a3f7-5ce83a59a675 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:19.645734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1287/2556] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:23.602528 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee565b52-9866-4680-8d76-c0d8932c80ea service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:23.673659 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee565b52-9866-4680-8d76-c0d8932c80ea service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:23.683145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:23.684010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:23.684293 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:23.684657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] PATH_INFO: `/v3/users/fee0ff3d3b534d90b0c824fe50a2e2b1` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:23.685938 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fee0ff3d3b534d90b0c824fe50a2e2b1)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:23.687936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:23.738708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fee0ff3d3b534d90b0c824fe50a2e2b1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:23.739259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fee0ff3d3b534d90b0c824fe50a2e2b1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:23.751673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fee0ff3d3b534d90b0c824fe50a2e2b1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:23.763209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fee0ff3d3b534d90b0c824fe50a2e2b1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:23.764366 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fee0ff3d3b534d90b0c824fe50a2e2b1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:23.787691 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1270/2557] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:54:23 2022] DELETE /identity/v3/users/fee0ff3d3b534d90b0c824fe50a2e2b1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:23.792250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baebd72-2996-4f07-8238-4c2be8d17c9e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:23.792822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baebd72-2996-4f07-8238-4c2be8d17c9e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:23.792822 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baebd72-2996-4f07-8238-4c2be8d17c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:23.803648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:23.822558 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5baebd72-2996-4f07-8238-4c2be8d17c9e None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:23.823832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5baebd72-2996-4f07-8238-4c2be8d17c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:23.895545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1288/2558] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:23.905912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:23.907431 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:23.968647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f71781-4172-4893-beb0-f083bbf34071 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:23.972342 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93634adf-0f1e-4869-bb68-eab46981e1f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:23.973100 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93634adf-0f1e-4869-bb68-eab46981e1f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:23.973429 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93634adf-0f1e-4869-bb68-eab46981e1f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:23.973783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93634adf-0f1e-4869-bb68-eab46981e1f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:23.984704 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93634adf-0f1e-4869-bb68-eab46981e1f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:23.986280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93634adf-0f1e-4869-bb68-eab46981e1f2 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:23.997977 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1271/2559] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:24.302581 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baebd72-2996-4f07-8238-4c2be8d17c9e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:24.304455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5baebd72-2996-4f07-8238-4c2be8d17c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:24.317486 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baebd72-2996-4f07-8238-4c2be8d17c9e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:24.324369 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:24.325290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:24.325668 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:24.325792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] PATH_INFO: `/v3/projects/77571cf4b2a24c9a9cdda407be5f09c6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:24.326937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] RBAC: Authorizing `identity:delete_project(project_id=77571cf4b2a24c9a9cdda407be5f09c6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:24.329029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:24.381989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77571cf4b2a24c9a9cdda407be5f09c6', 'request_id': 'req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:24.390577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77571cf4b2a24c9a9cdda407be5f09c6', 'request_id': 'req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:24.397093 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77571cf4b2a24c9a9cdda407be5f09c6', 'request_id': 'req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:24.411752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] The token cache is being invalidate because project 77571cf4b2a24c9a9cdda407be5f09c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:24.412042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:24.412711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:24.414914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1289/2560] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:24 2022] DELETE /identity/v3/projects/77571cf4b2a24c9a9cdda407be5f09c6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:24.419696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93634adf-0f1e-4869-bb68-eab46981e1f2 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:24.436163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93634adf-0f1e-4869-bb68-eab46981e1f2 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:24.455982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:24.500567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:24.501202 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:24.501487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:24.501767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] PATH_INFO: `/v3/users/1af339c95de7433caf45f97dc1e2bf50` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:24.503090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1af339c95de7433caf45f97dc1e2bf50)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:24.505046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:24.557855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1af339c95de7433caf45f97dc1e2bf50'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:24.558523 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1af339c95de7433caf45f97dc1e2bf50'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:24.565441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1af339c95de7433caf45f97dc1e2bf50'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:24.574050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1af339c95de7433caf45f97dc1e2bf50'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:24.574673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1af339c95de7433caf45f97dc1e2bf50'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:24.593247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1272/2561] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:54:24 2022] DELETE /identity/v3/users/1af339c95de7433caf45f97dc1e2bf50 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:24.999955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:25.001524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:25.014907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34230e4-4cf7-47a5-847b-26b0ac4ea41e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:25.089149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:25.089792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:25.090073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:25.090347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] PATH_INFO: `/v3/projects/cc565a17f3134974b95ccc93ea962efe` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:25.091471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc565a17f3134974b95ccc93ea962efe)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:25.093446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:25.132227 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:25.147928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc565a17f3134974b95ccc93ea962efe', 'request_id': 'req-5243f7ab-0bdb-4404-b75a-638506826010'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.155625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc565a17f3134974b95ccc93ea962efe', 'request_id': 'req-5243f7ab-0bdb-4404-b75a-638506826010'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.161764 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc565a17f3134974b95ccc93ea962efe', 'request_id': 'req-5243f7ab-0bdb-4404-b75a-638506826010'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.177318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] The token cache is being invalidate because project cc565a17f3134974b95ccc93ea962efe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:25.177661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.178146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.180058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1290/2562] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:24 2022] DELETE /identity/v3/projects/cc565a17f3134974b95ccc93ea962efe => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:25.184698 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:25.186305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:25.200145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753ec43-1c7f-4e63-bbfb-b7ad31e4b6e4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:25.262909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5248f-8633-4963-bd5b-6442729ddc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:25.263553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5248f-8633-4963-bd5b-6442729ddc8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:25.263867 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5248f-8633-4963-bd5b-6442729ddc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:25.264272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5248f-8633-4963-bd5b-6442729ddc8b admin admin] PATH_INFO: `/v3/users/d496f365e44e4cc2a7e6575d58239d72` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:25.265485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5248f-8633-4963-bd5b-6442729ddc8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d496f365e44e4cc2a7e6575d58239d72)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:25.267438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5248f-8633-4963-bd5b-6442729ddc8b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:25.315763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:25.322171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3db5248f-8633-4963-bd5b-6442729ddc8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd496f365e44e4cc2a7e6575d58239d72'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.323121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3db5248f-8633-4963-bd5b-6442729ddc8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd496f365e44e4cc2a7e6575d58239d72'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.331307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3db5248f-8633-4963-bd5b-6442729ddc8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd496f365e44e4cc2a7e6575d58239d72'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.339984 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3db5248f-8633-4963-bd5b-6442729ddc8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd496f365e44e4cc2a7e6575d58239d72'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.340756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3db5248f-8633-4963-bd5b-6442729ddc8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd496f365e44e4cc2a7e6575d58239d72'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.360702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1273/2563] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:54:25 2022] DELETE /identity/v3/users/d496f365e44e4cc2a7e6575d58239d72 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:25.661545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:25.662994 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:25.679604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243f7ab-0bdb-4404-b75a-638506826010 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:25.762427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:25.763083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:25.763358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:25.763627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] PATH_INFO: `/v3/projects/75df1313e8a04bb4bfce4ae213acdabd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:25.764937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] RBAC: Authorizing `identity:delete_project(project_id=75df1313e8a04bb4bfce4ae213acdabd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:25.766959 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:25.834442 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75df1313e8a04bb4bfce4ae213acdabd', 'request_id': 'req-a892e47a-fd97-4561-a46c-eab36400e17b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.840980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75df1313e8a04bb4bfce4ae213acdabd', 'request_id': 'req-a892e47a-fd97-4561-a46c-eab36400e17b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.846882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75df1313e8a04bb4bfce4ae213acdabd', 'request_id': 'req-a892e47a-fd97-4561-a46c-eab36400e17b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.867728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] The token cache is being invalidate because project 75df1313e8a04bb4bfce4ae213acdabd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:25.868692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.869519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a892e47a-fd97-4561-a46c-eab36400e17b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:25.871563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1291/2564] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:25 2022] DELETE /identity/v3/projects/75df1313e8a04bb4bfce4ae213acdabd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:25.877753 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08dbbb-8031-4e9c-963a-ae0928bf45df None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:25.878143 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08dbbb-8031-4e9c-963a-ae0928bf45df None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:25.879629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08dbbb-8031-4e9c-963a-ae0928bf45df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:25.900779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0d08dbbb-8031-4e9c-963a-ae0928bf45df None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:25.901694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0d08dbbb-8031-4e9c-963a-ae0928bf45df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:25.970359 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1274/2565] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:25.980052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b062b6-b5dc-471a-b8fe-e87fb786767f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:25.980342 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b062b6-b5dc-471a-b8fe-e87fb786767f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:25.980655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b062b6-b5dc-471a-b8fe-e87fb786767f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:26.001923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-72b062b6-b5dc-471a-b8fe-e87fb786767f None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:26.003153 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-72b062b6-b5dc-471a-b8fe-e87fb786767f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:26.015970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1292/2566] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:54:26.021006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08dbbb-8031-4e9c-963a-ae0928bf45df None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:26.031900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08dbbb-8031-4e9c-963a-ae0928bf45df None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:26.036951 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396ca52-7f14-439f-86c0-cc4ee8d5b716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:26.037684 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396ca52-7f14-439f-86c0-cc4ee8d5b716 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:26.037971 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396ca52-7f14-439f-86c0-cc4ee8d5b716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:26.038284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396ca52-7f14-439f-86c0-cc4ee8d5b716 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:26.038951 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396ca52-7f14-439f-86c0-cc4ee8d5b716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:54:26.039325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396ca52-7f14-439f-86c0-cc4ee8d5b716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:26.041970 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396ca52-7f14-439f-86c0-cc4ee8d5b716 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:26.054580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1275/2567] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:54:26.059557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b062b6-b5dc-471a-b8fe-e87fb786767f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:26.069770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b062b6-b5dc-471a-b8fe-e87fb786767f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:26.075168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dfac4-966b-4aee-a3ae-a79dbb76e4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:26.075781 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dfac4-966b-4aee-a3ae-a79dbb76e4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:26.076065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dfac4-966b-4aee-a3ae-a79dbb76e4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:26.076337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dfac4-966b-4aee-a3ae-a79dbb76e4c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:26.077063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2dfac4-966b-4aee-a3ae-a79dbb76e4c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:26.079013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2dfac4-966b-4aee-a3ae-a79dbb76e4c6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:26.110080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1293/2568] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:26 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:26.114525 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396ca52-7f14-439f-86c0-cc4ee8d5b716 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:26.128189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396ca52-7f14-439f-86c0-cc4ee8d5b716 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:26.191590 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29befb-44bf-4d21-983a-8e69fc54719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:26.192297 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29befb-44bf-4d21-983a-8e69fc54719f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:26.192534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29befb-44bf-4d21-983a-8e69fc54719f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:26.193014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29befb-44bf-4d21-983a-8e69fc54719f admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:26.193958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29befb-44bf-4d21-983a-8e69fc54719f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:26.195982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29befb-44bf-4d21-983a-8e69fc54719f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:26.235070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1276/2569] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:26 2022] POST /identity/v3/users => generated 282 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:54:26.236426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dfac4-966b-4aee-a3ae-a79dbb76e4c6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:26.247880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dfac4-966b-4aee-a3ae-a79dbb76e4c6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:26.252814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed4cc4-95fc-4e5d-b6ab-0a6d105d7463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:26.253423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed4cc4-95fc-4e5d-b6ab-0a6d105d7463 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:26.253828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed4cc4-95fc-4e5d-b6ab-0a6d105d7463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:26.254108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed4cc4-95fc-4e5d-b6ab-0a6d105d7463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:26.254759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed4cc4-95fc-4e5d-b6ab-0a6d105d7463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:26.256457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed4cc4-95fc-4e5d-b6ab-0a6d105d7463 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:26.267873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1294/2570] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:26.272120 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29befb-44bf-4d21-983a-8e69fc54719f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:26.287947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29befb-44bf-4d21-983a-8e69fc54719f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:26.295527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393d555-0f7d-4066-9c0d-c4d73a37f0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:26.296177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393d555-0f7d-4066-9c0d-c4d73a37f0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:26.296438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393d555-0f7d-4066-9c0d-c4d73a37f0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:26.297129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393d555-0f7d-4066-9c0d-c4d73a37f0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:26.297406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393d555-0f7d-4066-9c0d-c4d73a37f0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:26.299281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393d555-0f7d-4066-9c0d-c4d73a37f0b4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:26.310934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1277/2571] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:26.315410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed4cc4-95fc-4e5d-b6ab-0a6d105d7463 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:26.326681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed4cc4-95fc-4e5d-b6ab-0a6d105d7463 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:26.331799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f30f2a-68ad-438f-b74b-7fcd3916cfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:26.332585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f30f2a-68ad-438f-b74b-7fcd3916cfbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:26.332933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f30f2a-68ad-438f-b74b-7fcd3916cfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:26.333273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f30f2a-68ad-438f-b74b-7fcd3916cfbe admin admin] PATH_INFO: `/v3/projects/0ee5a24e009543a38e66a262fae177b9/users/b1e3571c1f31481b9985337fde1fc953/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:26.338689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:26.353531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f30f2a-68ad-438f-b74b-7fcd3916cfbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee5a24e009543a38e66a262fae177b9, user_id=b1e3571c1f31481b9985337fde1fc953, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:26.356262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f30f2a-68ad-438f-b74b-7fcd3916cfbe admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:26.370312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1295/2572] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:26 2022] PUT /identity/v3/projects/0ee5a24e009543a38e66a262fae177b9/users/b1e3571c1f31481b9985337fde1fc953/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:54:26.441043 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0e461-f6c3-4174-a083-1d044b4f67db None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:26.441409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0e461-f6c3-4174-a083-1d044b4f67db None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:26.441689 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0e461-f6c3-4174-a083-1d044b4f67db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:26.443519 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f30f2a-68ad-438f-b74b-7fcd3916cfbe admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:26.445553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d6f30f2a-68ad-438f-b74b-7fcd3916cfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:26.474628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5cd0e461-f6c3-4174-a083-1d044b4f67db None None] MFA Rules not processed for user `b1e3571c1f31481b9985337fde1fc953`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:26.476018 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5cd0e461-f6c3-4174-a083-1d044b4f67db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:26.493348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:26.536377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f30f2a-68ad-438f-b74b-7fcd3916cfbe admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:26.547328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928683fd-48e5-496c-b63d-05b42972548f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:26.547949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928683fd-48e5-496c-b63d-05b42972548f service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:26.548234 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928683fd-48e5-496c-b63d-05b42972548f service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:26.548517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928683fd-48e5-496c-b63d-05b42972548f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:26.550342 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-928683fd-48e5-496c-b63d-05b42972548f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:26.559521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1278/2573] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:26.564765 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928683fd-48e5-496c-b63d-05b42972548f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:26.566185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928683fd-48e5-496c-b63d-05b42972548f service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:26.572334 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0e461-f6c3-4174-a083-1d044b4f67db None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:26.574469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5cd0e461-f6c3-4174-a083-1d044b4f67db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:26.591114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1296/2574] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:26.656517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0e461-f6c3-4174-a083-1d044b4f67db None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:26.660125 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd252fcc-89bd-4ca5-8f29-607d6fde81a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:26.660774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd252fcc-89bd-4ca5-8f29-607d6fde81a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:26.661107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd252fcc-89bd-4ca5-8f29-607d6fde81a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:26.661438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd252fcc-89bd-4ca5-8f29-607d6fde81a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:26.671999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd252fcc-89bd-4ca5-8f29-607d6fde81a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:26.673817 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd252fcc-89bd-4ca5-8f29-607d6fde81a8 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:26.685445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1279/2575] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:54:36.574250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:36.679294 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=47 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:41.258933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928683fd-48e5-496c-b63d-05b42972548f service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:41.260901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-928683fd-48e5-496c-b63d-05b42972548f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:41.328527 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928683fd-48e5-496c-b63d-05b42972548f service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:41.332325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c3e4e-282c-470c-97f6-a93aebf72a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:41.332975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c3e4e-282c-470c-97f6-a93aebf72a3c service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:41.333250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c3e4e-282c-470c-97f6-a93aebf72a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:41.333517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c3e4e-282c-470c-97f6-a93aebf72a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:41.335202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a2c3e4e-282c-470c-97f6-a93aebf72a3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:41.399328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c3e4e-282c-470c-97f6-a93aebf72a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:41.400939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c3e4e-282c-470c-97f6-a93aebf72a3c service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:41.411269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1297/2576] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:42.338482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd252fcc-89bd-4ca5-8f29-607d6fde81a8 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:42.340408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dd252fcc-89bd-4ca5-8f29-607d6fde81a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:42.354009 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd252fcc-89bd-4ca5-8f29-607d6fde81a8 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:42.358513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb561628-1ec0-47bc-97cd-4260495d4285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:42.359360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb561628-1ec0-47bc-97cd-4260495d4285 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:42.359649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb561628-1ec0-47bc-97cd-4260495d4285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:42.359921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb561628-1ec0-47bc-97cd-4260495d4285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:42.361961 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-eb561628-1ec0-47bc-97cd-4260495d4285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:42.379204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb561628-1ec0-47bc-97cd-4260495d4285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:42.380774 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb561628-1ec0-47bc-97cd-4260495d4285 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:42.394343 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1280/2577] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:45.972401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c3e4e-282c-470c-97f6-a93aebf72a3c service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:45.974009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a2c3e4e-282c-470c-97f6-a93aebf72a3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:46.088146 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c3e4e-282c-470c-97f6-a93aebf72a3c service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:46.093674 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8778b9-6779-4c07-afd5-542f87618763 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:46.094971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8778b9-6779-4c07-afd5-542f87618763 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:46.094971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8778b9-6779-4c07-afd5-542f87618763 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:46.094971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8778b9-6779-4c07-afd5-542f87618763 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:46.097266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4b8778b9-6779-4c07-afd5-542f87618763 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:46.120314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8778b9-6779-4c07-afd5-542f87618763 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:46.125429 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8778b9-6779-4c07-afd5-542f87618763 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:46.126287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb561628-1ec0-47bc-97cd-4260495d4285 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:46.127762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-eb561628-1ec0-47bc-97cd-4260495d4285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:46.137854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1298/2578] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:46.143639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb561628-1ec0-47bc-97cd-4260495d4285 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:46.151467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139399af-e6b1-4f86-bae1-c225276ee5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:46.152524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139399af-e6b1-4f86-bae1-c225276ee5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:46.152909 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139399af-e6b1-4f86-bae1-c225276ee5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:46.153386 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139399af-e6b1-4f86-bae1-c225276ee5c8 admin admin] PATH_INFO: `/v3/users/bcd4602a40964a81931cdadc92d7f281` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:46.154723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139399af-e6b1-4f86-bae1-c225276ee5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd4602a40964a81931cdadc92d7f281)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:46.157591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139399af-e6b1-4f86-bae1-c225276ee5c8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:46.253967 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-139399af-e6b1-4f86-bae1-c225276ee5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd4602a40964a81931cdadc92d7f281'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:46.256756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-139399af-e6b1-4f86-bae1-c225276ee5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd4602a40964a81931cdadc92d7f281'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:46.280620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-139399af-e6b1-4f86-bae1-c225276ee5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd4602a40964a81931cdadc92d7f281'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:46.311527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-139399af-e6b1-4f86-bae1-c225276ee5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd4602a40964a81931cdadc92d7f281'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:46.312241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-139399af-e6b1-4f86-bae1-c225276ee5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd4602a40964a81931cdadc92d7f281'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:46.356146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1281/2579] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:54:46 2022] DELETE /identity/v3/users/bcd4602a40964a81931cdadc92d7f281 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:46.591401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8778b9-6779-4c07-afd5-542f87618763 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:46.604632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8778b9-6779-4c07-afd5-542f87618763 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:46.679451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:46.704278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:46.705406 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:46.705656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:46.706324 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] PATH_INFO: `/v3/projects/23acf169856c41d888a1d4c0e61cfee2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:46.707152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] RBAC: Authorizing `identity:delete_project(project_id=23acf169856c41d888a1d4c0e61cfee2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:46.713095 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:46.778649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23acf169856c41d888a1d4c0e61cfee2', 'request_id': 'req-186edde7-f4ee-4394-bc34-18fe32d5218e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:46.795149 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23acf169856c41d888a1d4c0e61cfee2', 'request_id': 'req-186edde7-f4ee-4394-bc34-18fe32d5218e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:46.802156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23acf169856c41d888a1d4c0e61cfee2', 'request_id': 'req-186edde7-f4ee-4394-bc34-18fe32d5218e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:46.822379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] The token cache is being invalidate because project 23acf169856c41d888a1d4c0e61cfee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:46.822708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:46.823496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:46.825729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1299/2580] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:46 2022] DELETE /identity/v3/projects/23acf169856c41d888a1d4c0e61cfee2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:46.831157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a720421-13f3-407b-b853-f5f10d85e727 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:46.831981 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a720421-13f3-407b-b853-f5f10d85e727 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:46.831981 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a720421-13f3-407b-b853-f5f10d85e727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:46.866089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4a720421-13f3-407b-b853-f5f10d85e727 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:46.868408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4a720421-13f3-407b-b853-f5f10d85e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:46.984534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1282/2581] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:46.989880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:46.991522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:46.993214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2640a-f329-4643-8989-ce3607b7985b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:46.993502 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2640a-f329-4643-8989-ce3607b7985b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:46.993779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2640a-f329-4643-8989-ce3607b7985b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:47.016318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-35c2640a-f329-4643-8989-ce3607b7985b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:47.017791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-35c2640a-f329-4643-8989-ce3607b7985b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:47.031960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1283/2582] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:54:47.033237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2640a-f329-4643-8989-ce3607b7985b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:47.043899 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2640a-f329-4643-8989-ce3607b7985b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:47.049934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac0730-f029-4ee7-8af0-6e01603067a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:47.050552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac0730-f029-4ee7-8af0-6e01603067a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:47.050831 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac0730-f029-4ee7-8af0-6e01603067a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:47.051102 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac0730-f029-4ee7-8af0-6e01603067a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:47.051876 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac0730-f029-4ee7-8af0-6e01603067a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:54:47.052194 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac0730-f029-4ee7-8af0-6e01603067a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:47.055348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac0730-f029-4ee7-8af0-6e01603067a7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:47.063625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186edde7-f4ee-4394-bc34-18fe32d5218e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:47.070552 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1284/2583] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:54:47.073466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132a2cf-4355-4773-a2e9-00ebd166bc90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:47.074222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132a2cf-4355-4773-a2e9-00ebd166bc90 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:47.074510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132a2cf-4355-4773-a2e9-00ebd166bc90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:47.074787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132a2cf-4355-4773-a2e9-00ebd166bc90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:47.075255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac0730-f029-4ee7-8af0-6e01603067a7 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:47.091300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac0730-f029-4ee7-8af0-6e01603067a7 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:47.091564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9132a2cf-4355-4773-a2e9-00ebd166bc90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:47.092458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd6af1-3361-465c-bc04-19aacb388d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:47.093157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd6af1-3361-465c-bc04-19aacb388d6d admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:47.093438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd6af1-3361-465c-bc04-19aacb388d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:47.093713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd6af1-3361-465c-bc04-19aacb388d6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:47.094353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd6af1-3361-465c-bc04-19aacb388d6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:47.096296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd6af1-3361-465c-bc04-19aacb388d6d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:47.129415 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1285/2584] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:47 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:54:47.134802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd6af1-3361-465c-bc04-19aacb388d6d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:47.164856 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:47.173608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132a2cf-4355-4773-a2e9-00ebd166bc90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:47.175847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132a2cf-4355-4773-a2e9-00ebd166bc90 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:47.189539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1300/2585] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:47.238229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd6af1-3361-465c-bc04-19aacb388d6d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:47.250439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a7d1b-c14e-4cd8-a783-d31ad853d437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:47.251091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a7d1b-c14e-4cd8-a783-d31ad853d437 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:47.251442 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a7d1b-c14e-4cd8-a783-d31ad853d437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:47.251730 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a7d1b-c14e-4cd8-a783-d31ad853d437 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:47.252373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a7d1b-c14e-4cd8-a783-d31ad853d437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:47.256626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a7d1b-c14e-4cd8-a783-d31ad853d437 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:47.307875 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1286/2586] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:47 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:47.312669 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132a2cf-4355-4773-a2e9-00ebd166bc90 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:47.324976 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132a2cf-4355-4773-a2e9-00ebd166bc90 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:47.330595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982c916-e40a-47e4-8126-2406799dbe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:47.331520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982c916-e40a-47e4-8126-2406799dbe80 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:47.331908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982c916-e40a-47e4-8126-2406799dbe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:47.332453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982c916-e40a-47e4-8126-2406799dbe80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:47.333528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982c916-e40a-47e4-8126-2406799dbe80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:47.335830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982c916-e40a-47e4-8126-2406799dbe80 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:47.349345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1301/2587] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:47.354569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a7d1b-c14e-4cd8-a783-d31ad853d437 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:47.365484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a7d1b-c14e-4cd8-a783-d31ad853d437 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:47.372775 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7414e6-aaae-4e4f-b085-483e5de98f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:47.374339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7414e6-aaae-4e4f-b085-483e5de98f0c admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:47.374858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7414e6-aaae-4e4f-b085-483e5de98f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:47.375804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7414e6-aaae-4e4f-b085-483e5de98f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:47.376751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7414e6-aaae-4e4f-b085-483e5de98f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:47.379292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7414e6-aaae-4e4f-b085-483e5de98f0c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:47.392312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1287/2588] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:47.396041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982c916-e40a-47e4-8126-2406799dbe80 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:47.415709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982c916-e40a-47e4-8126-2406799dbe80 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:47.421814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885c18d-61b2-4e4f-9839-f0399313181e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:47.422496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885c18d-61b2-4e4f-9839-f0399313181e admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:47.423069 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885c18d-61b2-4e4f-9839-f0399313181e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:47.423204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885c18d-61b2-4e4f-9839-f0399313181e admin admin] PATH_INFO: `/v3/projects/9146a0d909254dfd9fba5fd8f3883ad4/users/1013fab844d841d1bba713a69a05ec1b/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:47.430485 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:47.453513 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885c18d-61b2-4e4f-9839-f0399313181e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9146a0d909254dfd9fba5fd8f3883ad4, user_id=1013fab844d841d1bba713a69a05ec1b, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:47.456354 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885c18d-61b2-4e4f-9839-f0399313181e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:47.480156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1302/2589] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:47 2022] PUT /identity/v3/projects/9146a0d909254dfd9fba5fd8f3883ad4/users/1013fab844d841d1bba713a69a05ec1b/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:54:47.493428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f2c47-b592-4597-8bcd-534e3725dd9c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:47.493799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f2c47-b592-4597-8bcd-534e3725dd9c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:47.494180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f2c47-b592-4597-8bcd-534e3725dd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:47.527479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-397f2c47-b592-4597-8bcd-534e3725dd9c None None] MFA Rules not processed for user `1013fab844d841d1bba713a69a05ec1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:47.528764 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-397f2c47-b592-4597-8bcd-534e3725dd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:47.591192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885c18d-61b2-4e4f-9839-f0399313181e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:47.593312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4885c18d-61b2-4e4f-9839-f0399313181e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:47.637809 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1288/2590] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:47.651588 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f2c47-b592-4597-8bcd-534e3725dd9c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:47.654119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-397f2c47-b592-4597-8bcd-534e3725dd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:47.683999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885c18d-61b2-4e4f-9839-f0399313181e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:47.700656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:47.701606 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:47.701944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:47.702207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] PATH_INFO: `/v3/users/5de832888ffd4a69ae88cac6b632304e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:47.719496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de832888ffd4a69ae88cac6b632304e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:47.721597 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:47.778341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de832888ffd4a69ae88cac6b632304e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:47.786863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de832888ffd4a69ae88cac6b632304e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:47.787629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de832888ffd4a69ae88cac6b632304e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:47.788322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de832888ffd4a69ae88cac6b632304e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:47.798770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de832888ffd4a69ae88cac6b632304e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:47.802054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f2c47-b592-4597-8bcd-534e3725dd9c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:47.817230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0819e-77cb-4532-9e76-da41586eb8db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:47.818744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0819e-77cb-4532-9e76-da41586eb8db service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:47.818956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0819e-77cb-4532-9e76-da41586eb8db service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:47.819311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0819e-77cb-4532-9e76-da41586eb8db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:47.821979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fff0819e-77cb-4532-9e76-da41586eb8db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:47.827507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1303/2591] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:54:47 2022] DELETE /identity/v3/users/5de832888ffd4a69ae88cac6b632304e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:54:47.837495 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0819e-77cb-4532-9e76-da41586eb8db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:47.839295 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0819e-77cb-4532-9e76-da41586eb8db service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:47.851704 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1289/2592] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:54:48.095275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:48.097338 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:48.187960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:48.208540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c23b0-a68b-4ebf-a219-cc1b02f62404 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:48.213899 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:48.214949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:48.215234 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:48.215543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] PATH_INFO: `/v3/projects/b4ac4dc41f7e420d811483990e020f22` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:48.231350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4ac4dc41f7e420d811483990e020f22)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:48.233757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:48.320442 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4ac4dc41f7e420d811483990e020f22', 'request_id': 'req-d0d1a806-9b69-46d0-9a88-77f93157ab8a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:48.327309 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4ac4dc41f7e420d811483990e020f22', 'request_id': 'req-d0d1a806-9b69-46d0-9a88-77f93157ab8a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:48.333738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4ac4dc41f7e420d811483990e020f22', 'request_id': 'req-d0d1a806-9b69-46d0-9a88-77f93157ab8a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:48.347703 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] The token cache is being invalidate because project b4ac4dc41f7e420d811483990e020f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:54:48.348268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:48.349174 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d0d1a806-9b69-46d0-9a88-77f93157ab8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:54:48.352028 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1304/2593] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:54:48 2022] DELETE /identity/v3/projects/b4ac4dc41f7e420d811483990e020f22 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:48.356632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16925d55-25b0-4e37-b797-ff91ad39faa2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:48.357135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16925d55-25b0-4e37-b797-ff91ad39faa2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:48.357245 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16925d55-25b0-4e37-b797-ff91ad39faa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:48.392183 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-16925d55-25b0-4e37-b797-ff91ad39faa2 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:48.393510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-16925d55-25b0-4e37-b797-ff91ad39faa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:48.401766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:48.489860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1290/2594] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:48.498835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc7849-eac9-4742-9a86-6aae835eec7b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:48.500078 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc7849-eac9-4742-9a86-6aae835eec7b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:48.500569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc7849-eac9-4742-9a86-6aae835eec7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:48.524162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-61dc7849-eac9-4742-9a86-6aae835eec7b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:48.526607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-61dc7849-eac9-4742-9a86-6aae835eec7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:48.534788 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1305/2595] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:48.539338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16925d55-25b0-4e37-b797-ff91ad39faa2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:48.549687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16925d55-25b0-4e37-b797-ff91ad39faa2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:48.555408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b41b7d-10c0-4af0-94e6-491a8746a96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:48.556076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b41b7d-10c0-4af0-94e6-491a8746a96b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:48.556360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b41b7d-10c0-4af0-94e6-491a8746a96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:48.556674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b41b7d-10c0-4af0-94e6-491a8746a96b admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:48.557328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b41b7d-10c0-4af0-94e6-491a8746a96b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:54:48.557638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b41b7d-10c0-4af0-94e6-491a8746a96b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:48.559319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b41b7d-10c0-4af0-94e6-491a8746a96b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:48.575314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1291/2596] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:54:48.579626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc7849-eac9-4742-9a86-6aae835eec7b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:48.589539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc7849-eac9-4742-9a86-6aae835eec7b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:48.594387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0e988-05e9-495a-abf8-d20144a5d4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:48.594930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0e988-05e9-495a-abf8-d20144a5d4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:48.595210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0e988-05e9-495a-abf8-d20144a5d4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:48.595484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0e988-05e9-495a-abf8-d20144a5d4c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:48.596151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0e988-05e9-495a-abf8-d20144a5d4c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:48.598091 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0e988-05e9-495a-abf8-d20144a5d4c0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:48.629488 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1306/2597] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:48 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:54:48.634662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b41b7d-10c0-4af0-94e6-491a8746a96b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:48.644881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b41b7d-10c0-4af0-94e6-491a8746a96b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:48.709846 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275c4b7-7818-48f1-ad6c-011117431a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:48.710510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275c4b7-7818-48f1-ad6c-011117431a84 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:48.710793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275c4b7-7818-48f1-ad6c-011117431a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:48.711070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275c4b7-7818-48f1-ad6c-011117431a84 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:48.711785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275c4b7-7818-48f1-ad6c-011117431a84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:48.713804 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275c4b7-7818-48f1-ad6c-011117431a84 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:48.742877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1292/2598] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:48 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:54:48.747214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0e988-05e9-495a-abf8-d20144a5d4c0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:48.758162 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0e988-05e9-495a-abf8-d20144a5d4c0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:48.763474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c91fd-3c42-4301-bae5-89bc0585406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:48.764075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c91fd-3c42-4301-bae5-89bc0585406d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:48.764350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c91fd-3c42-4301-bae5-89bc0585406d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:48.764649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c91fd-3c42-4301-bae5-89bc0585406d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:48.765291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c91fd-3c42-4301-bae5-89bc0585406d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:48.766861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c91fd-3c42-4301-bae5-89bc0585406d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:48.777825 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1307/2599] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:48.781833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275c4b7-7818-48f1-ad6c-011117431a84 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:48.792689 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275c4b7-7818-48f1-ad6c-011117431a84 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:48.797759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40961049-a418-4ce9-b582-84176df7fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:48.798433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40961049-a418-4ce9-b582-84176df7fb8f admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:48.798730 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40961049-a418-4ce9-b582-84176df7fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:48.799005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40961049-a418-4ce9-b582-84176df7fb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:48.799653 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40961049-a418-4ce9-b582-84176df7fb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:48.801452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40961049-a418-4ce9-b582-84176df7fb8f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:48.813690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1293/2600] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:54:48.818271 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c91fd-3c42-4301-bae5-89bc0585406d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:48.829125 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c91fd-3c42-4301-bae5-89bc0585406d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:48.834365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adccdb-b1a8-4132-8654-236ac538e05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:48.835025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adccdb-b1a8-4132-8654-236ac538e05a admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:48.835378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adccdb-b1a8-4132-8654-236ac538e05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:48.835656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adccdb-b1a8-4132-8654-236ac538e05a admin admin] PATH_INFO: `/v3/projects/ca915d45a4934088a389750b6c923926/users/ea2f82c9939d40898d1206151a2e2dc7/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:48.853979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adccdb-b1a8-4132-8654-236ac538e05a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca915d45a4934088a389750b6c923926, user_id=ea2f82c9939d40898d1206151a2e2dc7, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:48.858140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adccdb-b1a8-4132-8654-236ac538e05a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:48.876257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1308/2601] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:48 2022] PUT /identity/v3/projects/ca915d45a4934088a389750b6c923926/users/ea2f82c9939d40898d1206151a2e2dc7/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:48.886952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7751f88-6e06-4825-ba3e-e86b0b435608 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:48.887246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7751f88-6e06-4825-ba3e-e86b0b435608 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:48.887531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7751f88-6e06-4825-ba3e-e86b0b435608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:48.916793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c7751f88-6e06-4825-ba3e-e86b0b435608 None None] MFA Rules not processed for user `ea2f82c9939d40898d1206151a2e2dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:48.918445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c7751f88-6e06-4825-ba3e-e86b0b435608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:49.011631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1294/2602] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:49.017029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adccdb-b1a8-4132-8654-236ac538e05a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:49.087755 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adccdb-b1a8-4132-8654-236ac538e05a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:49.092541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5118f-940c-4b95-aa2e-74037082bb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:49.093165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5118f-940c-4b95-aa2e-74037082bb57 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:49.093446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5118f-940c-4b95-aa2e-74037082bb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:49.093717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5118f-940c-4b95-aa2e-74037082bb57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:49.094386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5118f-940c-4b95-aa2e-74037082bb57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:49.096356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5118f-940c-4b95-aa2e-74037082bb57 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:49.115133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:49.135686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1309/2603] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:54:49 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:54:49.137099 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7751f88-6e06-4825-ba3e-e86b0b435608 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:49.148975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7751f88-6e06-4825-ba3e-e86b0b435608 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:49.152994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:54:49.218419 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06d335-cff0-44bf-be6c-c4d757081a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:49.219050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d335-cff0-44bf-be6c-c4d757081a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:49.219336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d335-cff0-44bf-be6c-c4d757081a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:49.219605 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d335-cff0-44bf-be6c-c4d757081a7f admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:49.220258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06d335-cff0-44bf-be6c-c4d757081a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:49.222309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06d335-cff0-44bf-be6c-c4d757081a7f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:49.252513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1295/2604] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:54:49 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:54:49.259926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5118f-940c-4b95-aa2e-74037082bb57 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:49.274116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5118f-940c-4b95-aa2e-74037082bb57 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:49.280938 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f0c2b-c7ab-4570-b915-2e25702a6f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:49.281710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f0c2b-c7ab-4570-b915-2e25702a6f33 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:49.281882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f0c2b-c7ab-4570-b915-2e25702a6f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:49.282105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f0c2b-c7ab-4570-b915-2e25702a6f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:49.282809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f0c2b-c7ab-4570-b915-2e25702a6f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:49.284417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f0c2b-c7ab-4570-b915-2e25702a6f33 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:49.298022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1310/2605] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:54:49.302658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06d335-cff0-44bf-be6c-c4d757081a7f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:49.318321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06d335-cff0-44bf-be6c-c4d757081a7f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:49.324988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598ad31-4574-4d18-aeeb-8b35c29cf0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:49.325785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598ad31-4574-4d18-aeeb-8b35c29cf0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:49.326138 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598ad31-4574-4d18-aeeb-8b35c29cf0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:49.326484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598ad31-4574-4d18-aeeb-8b35c29cf0ee admin admin] PATH_INFO: `/v3/projects/583e18fadca54c839368e74efa8b5f16/users/0d7e984fd774456f958c4100cee9e3c6/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:49.346547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598ad31-4574-4d18-aeeb-8b35c29cf0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=583e18fadca54c839368e74efa8b5f16, user_id=0d7e984fd774456f958c4100cee9e3c6, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:49.349292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598ad31-4574-4d18-aeeb-8b35c29cf0ee admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:49.363553 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1296/2606] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:49 2022] PUT /identity/v3/projects/583e18fadca54c839368e74efa8b5f16/users/0d7e984fd774456f958c4100cee9e3c6/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:54:49.368044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f0c2b-c7ab-4570-b915-2e25702a6f33 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:49.380168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f0c2b-c7ab-4570-b915-2e25702a6f33 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:49.440365 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d630adf-b67a-40b6-b295-9b45c9308614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:49.441197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d630adf-b67a-40b6-b295-9b45c9308614 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:49.441586 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d630adf-b67a-40b6-b295-9b45c9308614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:49.441907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d630adf-b67a-40b6-b295-9b45c9308614 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:49.443026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d630adf-b67a-40b6-b295-9b45c9308614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:49.445198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d630adf-b67a-40b6-b295-9b45c9308614 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:49.457344 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1311/2607] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:54:49.462209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598ad31-4574-4d18-aeeb-8b35c29cf0ee admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:49.474786 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598ad31-4574-4d18-aeeb-8b35c29cf0ee admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:49.506495 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1be888-9349-4d66-a7c8-9df5f0fb331d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:49.509230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1be888-9349-4d66-a7c8-9df5f0fb331d admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:49.509780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1be888-9349-4d66-a7c8-9df5f0fb331d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:49.510132 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1be888-9349-4d66-a7c8-9df5f0fb331d admin admin] PATH_INFO: `/v3/projects/583e18fadca54c839368e74efa8b5f16/users/0d7e984fd774456f958c4100cee9e3c6/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:49.512299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1be888-9349-4d66-a7c8-9df5f0fb331d admin admin] RBAC: Authorizing `identity:create_grant(project_id=583e18fadca54c839368e74efa8b5f16, user_id=0d7e984fd774456f958c4100cee9e3c6, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:49.516074 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1be888-9349-4d66-a7c8-9df5f0fb331d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:49.532280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1297/2608] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:54:49 2022] PUT /identity/v3/projects/583e18fadca54c839368e74efa8b5f16/users/0d7e984fd774456f958c4100cee9e3c6/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:54:49.545624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b4447-149e-43dd-8c8c-650a1a52f36b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:49.545906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b4447-149e-43dd-8c8c-650a1a52f36b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:49.546177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b4447-149e-43dd-8c8c-650a1a52f36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:49.609010 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8b9b4447-149e-43dd-8c8c-650a1a52f36b None None] MFA Rules not processed for user `0d7e984fd774456f958c4100cee9e3c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:54:49.610638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8b9b4447-149e-43dd-8c8c-650a1a52f36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:49.738198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1312/2609] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:54:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:54:49.750604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1be888-9349-4d66-a7c8-9df5f0fb331d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:49.752633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1d1be888-9349-4d66-a7c8-9df5f0fb331d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:54:49.889397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1be888-9349-4d66-a7c8-9df5f0fb331d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:49.893829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea22b7-5a17-45b2-911b-8858cc9d9549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:49.894447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea22b7-5a17-45b2-911b-8858cc9d9549 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:49.894728 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea22b7-5a17-45b2-911b-8858cc9d9549 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:49.895000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea22b7-5a17-45b2-911b-8858cc9d9549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:49.921050 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea22b7-5a17-45b2-911b-8858cc9d9549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:49.922598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea22b7-5a17-45b2-911b-8858cc9d9549 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:49.938688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1298/2610] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:50.033068 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b4447-149e-43dd-8c8c-650a1a52f36b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:54:50.043972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b4447-149e-43dd-8c8c-650a1a52f36b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:54:50.047820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b1761-0b7c-4ccc-8cf0-43476a45260d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:54:50.048457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b1761-0b7c-4ccc-8cf0-43476a45260d service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:54:50.048768 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b1761-0b7c-4ccc-8cf0-43476a45260d service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:54:50.049088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b1761-0b7c-4ccc-8cf0-43476a45260d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:54:50.125117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b1761-0b7c-4ccc-8cf0-43476a45260d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:54:50.127424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b1761-0b7c-4ccc-8cf0-43476a45260d service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:54:50.137940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1313/2611] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:54:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:54:59.928979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:00.131802 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:04.032913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea22b7-5a17-45b2-911b-8858cc9d9549 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:04.033951 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c8ea22b7-5a17-45b2-911b-8858cc9d9549 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:04.100973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea22b7-5a17-45b2-911b-8858cc9d9549 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:04.105599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:04.106229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:04.106510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:04.106786 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] PATH_INFO: `/v3/users/1013fab844d841d1bba713a69a05ec1b` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:04.107921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1013fab844d841d1bba713a69a05ec1b)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:04.109873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:04.158541 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1013fab844d841d1bba713a69a05ec1b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:04.159212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1013fab844d841d1bba713a69a05ec1b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:04.167877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1013fab844d841d1bba713a69a05ec1b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:04.177291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1013fab844d841d1bba713a69a05ec1b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:04.177912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1013fab844d841d1bba713a69a05ec1b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:04.197180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1299/2612] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:04 2022] DELETE /identity/v3/users/1013fab844d841d1bba713a69a05ec1b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:04.201574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8ef67-6040-44b5-af65-8fc5979b9ee8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:04.202056 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8ef67-6040-44b5-af65-8fc5979b9ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:04.202298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8ef67-6040-44b5-af65-8fc5979b9ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:04.221860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3fa8ef67-6040-44b5-af65-8fc5979b9ee8 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:04.223044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3fa8ef67-6040-44b5-af65-8fc5979b9ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:04.292009 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1314/2613] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:04.302036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:04.304012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:04.375199 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963caac-67d5-45de-a776-d0e8def735dd admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:04.379521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2f1fe-b117-49ea-a30a-2bf83a6a1692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:04.380246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2f1fe-b117-49ea-a30a-2bf83a6a1692 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:04.380556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2f1fe-b117-49ea-a30a-2bf83a6a1692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:04.380832 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2f1fe-b117-49ea-a30a-2bf83a6a1692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:04.391173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2f1fe-b117-49ea-a30a-2bf83a6a1692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:04.392925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2f1fe-b117-49ea-a30a-2bf83a6a1692 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:04.404073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1300/2614] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:04.675903 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8ef67-6040-44b5-af65-8fc5979b9ee8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:04.677736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3fa8ef67-6040-44b5-af65-8fc5979b9ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:04.691496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8ef67-6040-44b5-af65-8fc5979b9ee8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:04.698117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:04.698831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:04.699184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:04.699466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] PATH_INFO: `/v3/projects/9146a0d909254dfd9fba5fd8f3883ad4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:04.700801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9146a0d909254dfd9fba5fd8f3883ad4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:04.702729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:04.749254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9146a0d909254dfd9fba5fd8f3883ad4', 'request_id': 'req-b64cbf9e-cc28-4a96-8120-275dd0f9b920'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:04.755198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9146a0d909254dfd9fba5fd8f3883ad4', 'request_id': 'req-b64cbf9e-cc28-4a96-8120-275dd0f9b920'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:04.761107 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9146a0d909254dfd9fba5fd8f3883ad4', 'request_id': 'req-b64cbf9e-cc28-4a96-8120-275dd0f9b920'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:04.774125 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] The token cache is being invalidate because project 9146a0d909254dfd9fba5fd8f3883ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:04.774646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:04.775734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b64cbf9e-cc28-4a96-8120-275dd0f9b920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:04.777949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1315/2615] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:04 2022] DELETE /identity/v3/projects/9146a0d909254dfd9fba5fd8f3883ad4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:04.784662 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95796b-4053-4a65-be35-750718e8a941 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:04.784955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95796b-4053-4a65-be35-750718e8a941 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:04.785235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95796b-4053-4a65-be35-750718e8a941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:04.806885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2c95796b-4053-4a65-be35-750718e8a941 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:04.808113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2c95796b-4053-4a65-be35-750718e8a941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:04.825941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:04.869645 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1301/2616] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:04.877387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32482a8f-43d4-4e6c-9e49-3b6aa1b2bc1b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:04.877678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32482a8f-43d4-4e6c-9e49-3b6aa1b2bc1b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:04.877956 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32482a8f-43d4-4e6c-9e49-3b6aa1b2bc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:04.898563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-32482a8f-43d4-4e6c-9e49-3b6aa1b2bc1b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:04.899771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-32482a8f-43d4-4e6c-9e49-3b6aa1b2bc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:04.907585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1316/2617] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:04.912026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95796b-4053-4a65-be35-750718e8a941 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:04.922173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95796b-4053-4a65-be35-750718e8a941 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:04.926763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a67273-eaf5-4b4f-a46d-73a904960c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:04.927365 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a67273-eaf5-4b4f-a46d-73a904960c7b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:04.927652 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a67273-eaf5-4b4f-a46d-73a904960c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:04.927930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a67273-eaf5-4b4f-a46d-73a904960c7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:04.929394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a67273-eaf5-4b4f-a46d-73a904960c7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:55:04.929704 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a67273-eaf5-4b4f-a46d-73a904960c7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:04.931306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a67273-eaf5-4b4f-a46d-73a904960c7b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:04.943083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1302/2618] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:55:04.948564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32482a8f-43d4-4e6c-9e49-3b6aa1b2bc1b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:04.957946 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32482a8f-43d4-4e6c-9e49-3b6aa1b2bc1b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:04.962596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfe958-8417-4982-bf46-fd608b04e971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:04.963186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfe958-8417-4982-bf46-fd608b04e971 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:04.963466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfe958-8417-4982-bf46-fd608b04e971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:04.963736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfe958-8417-4982-bf46-fd608b04e971 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:04.964355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfe958-8417-4982-bf46-fd608b04e971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:04.966290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfe958-8417-4982-bf46-fd608b04e971 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:04.994637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1317/2619] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:04 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:04.998883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a67273-eaf5-4b4f-a46d-73a904960c7b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:05.009765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a67273-eaf5-4b4f-a46d-73a904960c7b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:05.065802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4854f92-a8bf-45f2-b2ed-50028ee8615e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:05.066492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854f92-a8bf-45f2-b2ed-50028ee8615e admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:05.066675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854f92-a8bf-45f2-b2ed-50028ee8615e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:05.066955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854f92-a8bf-45f2-b2ed-50028ee8615e admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:05.068125 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4854f92-a8bf-45f2-b2ed-50028ee8615e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:05.070459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4854f92-a8bf-45f2-b2ed-50028ee8615e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:05.091065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1303/2620] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:04 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:55:05.095223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfe958-8417-4982-bf46-fd608b04e971 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:05.105455 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfe958-8417-4982-bf46-fd608b04e971 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:05.110041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c6ece-90a6-4265-8c9e-0775a139e46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:05.110635 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c6ece-90a6-4265-8c9e-0775a139e46b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:05.110921 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c6ece-90a6-4265-8c9e-0775a139e46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:05.111198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c6ece-90a6-4265-8c9e-0775a139e46b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:05.111838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c6ece-90a6-4265-8c9e-0775a139e46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:05.113474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c6ece-90a6-4265-8c9e-0775a139e46b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:05.124074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1318/2621] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:05.127964 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4854f92-a8bf-45f2-b2ed-50028ee8615e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:05.138898 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4854f92-a8bf-45f2-b2ed-50028ee8615e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:05.143538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe334b3-6143-4568-9456-f92feb2d26e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:05.144127 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe334b3-6143-4568-9456-f92feb2d26e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:05.144408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe334b3-6143-4568-9456-f92feb2d26e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:05.144723 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe334b3-6143-4568-9456-f92feb2d26e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:05.145363 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe334b3-6143-4568-9456-f92feb2d26e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:05.147016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe334b3-6143-4568-9456-f92feb2d26e3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:05.157781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1304/2622] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:05.161797 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c6ece-90a6-4265-8c9e-0775a139e46b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:05.172288 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c6ece-90a6-4265-8c9e-0775a139e46b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:05.176863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f83ff2-68ac-4608-b6ab-40d7f43285c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:05.177509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f83ff2-68ac-4608-b6ab-40d7f43285c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:05.177784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f83ff2-68ac-4608-b6ab-40d7f43285c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:05.178572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f83ff2-68ac-4608-b6ab-40d7f43285c2 admin admin] PATH_INFO: `/v3/projects/7562d5717ca94c509dd7ac6aea987565/users/e55afbd435694ea0b9ea0d1a19a4abdb/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:05.193448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f83ff2-68ac-4608-b6ab-40d7f43285c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7562d5717ca94c509dd7ac6aea987565, user_id=e55afbd435694ea0b9ea0d1a19a4abdb, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:05.196749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f83ff2-68ac-4608-b6ab-40d7f43285c2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:05.210804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1319/2623] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:05 2022] PUT /identity/v3/projects/7562d5717ca94c509dd7ac6aea987565/users/e55afbd435694ea0b9ea0d1a19a4abdb/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:05.222213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de670e1-2f9c-48e5-b2e0-22ae630edf89 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:05.222571 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de670e1-2f9c-48e5-b2e0-22ae630edf89 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:05.222860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de670e1-2f9c-48e5-b2e0-22ae630edf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:05.252041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8de670e1-2f9c-48e5-b2e0-22ae630edf89 None None] MFA Rules not processed for user `e55afbd435694ea0b9ea0d1a19a4abdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:05.253299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8de670e1-2f9c-48e5-b2e0-22ae630edf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:05.320068 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1305/2624] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:05.331790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f83ff2-68ac-4608-b6ab-40d7f43285c2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:05.333528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f6f83ff2-68ac-4608-b6ab-40d7f43285c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:05.356049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:05.419156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f83ff2-68ac-4608-b6ab-40d7f43285c2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:05.424741 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb360a-3bda-4e72-b00d-f7e5ab74330c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:05.425377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb360a-3bda-4e72-b00d-f7e5ab74330c service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:05.425659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb360a-3bda-4e72-b00d-f7e5ab74330c service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:05.426550 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb360a-3bda-4e72-b00d-f7e5ab74330c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:05.435484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb360a-3bda-4e72-b00d-f7e5ab74330c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:05.437081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb360a-3bda-4e72-b00d-f7e5ab74330c service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:05.447048 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1320/2625] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:05.724334 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de670e1-2f9c-48e5-b2e0-22ae630edf89 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:05.745065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:05.792623 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de670e1-2f9c-48e5-b2e0-22ae630edf89 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:05.797299 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:05.797940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:05.798241 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:05.798513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] PATH_INFO: `/v3/users/e55afbd435694ea0b9ea0d1a19a4abdb` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:05.799866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] RBAC: Authorizing `identity:delete_user(user_id=e55afbd435694ea0b9ea0d1a19a4abdb)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:05.802456 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:05.852254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e55afbd435694ea0b9ea0d1a19a4abdb'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:05.852912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e55afbd435694ea0b9ea0d1a19a4abdb'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:05.859934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e55afbd435694ea0b9ea0d1a19a4abdb'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:05.867707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e55afbd435694ea0b9ea0d1a19a4abdb'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:05.868440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e55afbd435694ea0b9ea0d1a19a4abdb'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:05.887071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1306/2626] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:05 2022] DELETE /identity/v3/users/e55afbd435694ea0b9ea0d1a19a4abdb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:05.892059 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e2a23-d033-44ba-83f1-19e572be2d46 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:05.892381 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e2a23-d033-44ba-83f1-19e572be2d46 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:05.892686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e2a23-d033-44ba-83f1-19e572be2d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:05.914630 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cf4e2a23-d033-44ba-83f1-19e572be2d46 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:05.915823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf4e2a23-d033-44ba-83f1-19e572be2d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:05.976576 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1321/2627] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:05.986216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:05.987631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:06.047712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b82bb-9f64-4380-8930-0a20cd26d0de admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:06.051345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f6e7b-5db3-443e-a469-321e03578328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:06.051942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f6e7b-5db3-443e-a469-321e03578328 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.052221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f6e7b-5db3-443e-a469-321e03578328 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.052506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f6e7b-5db3-443e-a469-321e03578328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.061888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f6e7b-5db3-443e-a469-321e03578328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:06.063110 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f6e7b-5db3-443e-a469-321e03578328 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:06.073237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1307/2628] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:06.347898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e2a23-d033-44ba-83f1-19e572be2d46 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:06.350321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf4e2a23-d033-44ba-83f1-19e572be2d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:06.364124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e2a23-d033-44ba-83f1-19e572be2d46 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:06.369577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:06.370143 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.370423 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.370692 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] PATH_INFO: `/v3/projects/7562d5717ca94c509dd7ac6aea987565` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.371804 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7562d5717ca94c509dd7ac6aea987565)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:06.373725 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:06.420743 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7562d5717ca94c509dd7ac6aea987565', 'request_id': 'req-c2104bc5-1873-4856-8075-3b2fd7d285c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:06.426724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7562d5717ca94c509dd7ac6aea987565', 'request_id': 'req-c2104bc5-1873-4856-8075-3b2fd7d285c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:06.432960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7562d5717ca94c509dd7ac6aea987565', 'request_id': 'req-c2104bc5-1873-4856-8075-3b2fd7d285c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:06.446512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] The token cache is being invalidate because project 7562d5717ca94c509dd7ac6aea987565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:06.446799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:06.447489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c2104bc5-1873-4856-8075-3b2fd7d285c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:06.449238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1322/2629] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:06 2022] DELETE /identity/v3/projects/7562d5717ca94c509dd7ac6aea987565 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:06.454079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdeae3-544f-49f2-a018-67f94beb1847 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.454375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdeae3-544f-49f2-a018-67f94beb1847 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.454788 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdeae3-544f-49f2-a018-67f94beb1847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.475518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-15fdeae3-544f-49f2-a018-67f94beb1847 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:06.476253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-15fdeae3-544f-49f2-a018-67f94beb1847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:06.512051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:06.539623 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1308/2630] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:06.547489 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5998b0-7907-4ca4-8435-6ebbc2497906 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.547773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5998b0-7907-4ca4-8435-6ebbc2497906 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.548042 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5998b0-7907-4ca4-8435-6ebbc2497906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.568306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ed5998b0-7907-4ca4-8435-6ebbc2497906 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:06.570081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ed5998b0-7907-4ca4-8435-6ebbc2497906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:06.578264 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1323/2631] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:06.582635 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdeae3-544f-49f2-a018-67f94beb1847 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:06.592580 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdeae3-544f-49f2-a018-67f94beb1847 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:06.599019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6be8cd-d93b-414e-acf7-aae3575965eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:06.599630 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6be8cd-d93b-414e-acf7-aae3575965eb admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.599968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6be8cd-d93b-414e-acf7-aae3575965eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.600263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6be8cd-d93b-414e-acf7-aae3575965eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.600957 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6be8cd-d93b-414e-acf7-aae3575965eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:55:06.601260 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6be8cd-d93b-414e-acf7-aae3575965eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:06.602866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6be8cd-d93b-414e-acf7-aae3575965eb admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:06.614456 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1309/2632] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:55:06.619211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5998b0-7907-4ca4-8435-6ebbc2497906 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:06.628203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5998b0-7907-4ca4-8435-6ebbc2497906 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:06.633355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd4645-ab46-4288-a116-03b101c83d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:06.633936 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd4645-ab46-4288-a116-03b101c83d2f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.634214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd4645-ab46-4288-a116-03b101c83d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.634482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd4645-ab46-4288-a116-03b101c83d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.635128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd4645-ab46-4288-a116-03b101c83d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:06.637004 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd4645-ab46-4288-a116-03b101c83d2f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:06.658942 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:06.669836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1324/2633] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:06 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:06.670966 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6be8cd-d93b-414e-acf7-aae3575965eb admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:06.681742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6be8cd-d93b-414e-acf7-aae3575965eb admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:06.741715 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636279f2-236c-4e1d-bc36-1aad563a6980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:06.742338 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636279f2-236c-4e1d-bc36-1aad563a6980 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.742620 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636279f2-236c-4e1d-bc36-1aad563a6980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.742896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636279f2-236c-4e1d-bc36-1aad563a6980 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.743537 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636279f2-236c-4e1d-bc36-1aad563a6980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:06.745544 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636279f2-236c-4e1d-bc36-1aad563a6980 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:06.768300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1310/2634] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:06 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:06.772711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd4645-ab46-4288-a116-03b101c83d2f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:06.783466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd4645-ab46-4288-a116-03b101c83d2f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:06.789325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65726fdf-afcf-4808-bdaf-3a4251068aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:06.794218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65726fdf-afcf-4808-bdaf-3a4251068aeb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.794218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65726fdf-afcf-4808-bdaf-3a4251068aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.794218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65726fdf-afcf-4808-bdaf-3a4251068aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.794218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65726fdf-afcf-4808-bdaf-3a4251068aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:06.794218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65726fdf-afcf-4808-bdaf-3a4251068aeb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:06.807479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1325/2635] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:06.811879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636279f2-236c-4e1d-bc36-1aad563a6980 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:06.822440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636279f2-236c-4e1d-bc36-1aad563a6980 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:06.827214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41779bc1-af4d-4e4b-bcda-79f589cdc096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:06.827822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779bc1-af4d-4e4b-bcda-79f589cdc096 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.828108 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779bc1-af4d-4e4b-bcda-79f589cdc096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.828387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779bc1-af4d-4e4b-bcda-79f589cdc096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.829084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41779bc1-af4d-4e4b-bcda-79f589cdc096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:06.830665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41779bc1-af4d-4e4b-bcda-79f589cdc096 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:06.841887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1311/2636] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:55:06.846257 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65726fdf-afcf-4808-bdaf-3a4251068aeb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:06.857424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65726fdf-afcf-4808-bdaf-3a4251068aeb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:06.862268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a445cf2-29ae-420c-82b0-208f3c369b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:06.863005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a445cf2-29ae-420c-82b0-208f3c369b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.863305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a445cf2-29ae-420c-82b0-208f3c369b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.863569 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a445cf2-29ae-420c-82b0-208f3c369b79 admin admin] PATH_INFO: `/v3/projects/6b8afeb4e04a44ec858ce5404fd18757/users/9793bff878db44f1923824b297e2285c/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.880392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a445cf2-29ae-420c-82b0-208f3c369b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8afeb4e04a44ec858ce5404fd18757, user_id=9793bff878db44f1923824b297e2285c, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:06.883439 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a445cf2-29ae-420c-82b0-208f3c369b79 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:06.889943 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41779bc1-af4d-4e4b-bcda-79f589cdc096 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:06.891845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-41779bc1-af4d-4e4b-bcda-79f589cdc096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:06.897783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1326/2637] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:06 2022] PUT /identity/v3/projects/6b8afeb4e04a44ec858ce5404fd18757/users/9793bff878db44f1923824b297e2285c/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:06.908082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76a99-42a0-4371-afd7-1e39ddcc678c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.908363 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76a99-42a0-4371-afd7-1e39ddcc678c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.908667 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76a99-42a0-4371-afd7-1e39ddcc678c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.909440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41779bc1-af4d-4e4b-bcda-79f589cdc096 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:06.936616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3fa76a99-42a0-4371-afd7-1e39ddcc678c None None] MFA Rules not processed for user `9793bff878db44f1923824b297e2285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:06.937738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3fa76a99-42a0-4371-afd7-1e39ddcc678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:06.969573 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530053b8-0149-4609-8165-4c390dc92d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:06.970210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530053b8-0149-4609-8165-4c390dc92d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:06.970493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530053b8-0149-4609-8165-4c390dc92d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:06.970772 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530053b8-0149-4609-8165-4c390dc92d4d admin admin] PATH_INFO: `/v3/users/ea2f82c9939d40898d1206151a2e2dc7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:06.971916 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530053b8-0149-4609-8165-4c390dc92d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2f82c9939d40898d1206151a2e2dc7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:06.973873 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530053b8-0149-4609-8165-4c390dc92d4d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:07.004104 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1327/2638] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:07.009259 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa76a99-42a0-4371-afd7-1e39ddcc678c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:07.020129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa76a99-42a0-4371-afd7-1e39ddcc678c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:07.024876 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77ad3e-7704-4d85-8dad-c9f8a1c7efad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:07.025539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77ad3e-7704-4d85-8dad-c9f8a1c7efad admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.025826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77ad3e-7704-4d85-8dad-c9f8a1c7efad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.026095 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77ad3e-7704-4d85-8dad-c9f8a1c7efad admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.026763 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77ad3e-7704-4d85-8dad-c9f8a1c7efad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:07.028867 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77ad3e-7704-4d85-8dad-c9f8a1c7efad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:07.029790 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-530053b8-0149-4609-8165-4c390dc92d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2f82c9939d40898d1206151a2e2dc7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.030720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-530053b8-0149-4609-8165-4c390dc92d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2f82c9939d40898d1206151a2e2dc7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.033149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:07.039707 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-530053b8-0149-4609-8165-4c390dc92d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2f82c9939d40898d1206151a2e2dc7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.048477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-530053b8-0149-4609-8165-4c390dc92d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2f82c9939d40898d1206151a2e2dc7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.049174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-530053b8-0149-4609-8165-4c390dc92d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2f82c9939d40898d1206151a2e2dc7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.058813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1328/2639] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:07 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:55:07.062887 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77ad3e-7704-4d85-8dad-c9f8a1c7efad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:07.064463 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fb77ad3e-7704-4d85-8dad-c9f8a1c7efad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:07.068124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1312/2640] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:06 2022] DELETE /identity/v3/users/ea2f82c9939d40898d1206151a2e2dc7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:55:07.072610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5e5aa-2e5b-466e-a0ec-cd51ae09a0d9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.072903 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5e5aa-2e5b-466e-a0ec-cd51ae09a0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.073172 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5e5aa-2e5b-466e-a0ec-cd51ae09a0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.091543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b2b5e5aa-2e5b-466e-a0ec-cd51ae09a0d9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:07.092771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b2b5e5aa-2e5b-466e-a0ec-cd51ae09a0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:07.131759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77ad3e-7704-4d85-8dad-c9f8a1c7efad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:07.153551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1313/2641] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:07.164024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5e5aa-2e5b-466e-a0ec-cd51ae09a0d9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:07.165426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b2b5e5aa-2e5b-466e-a0ec-cd51ae09a0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:07.173289 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:07.194177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3904e5-deca-45db-a585-9c7cd632ebee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:07.194878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3904e5-deca-45db-a585-9c7cd632ebee admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.195159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3904e5-deca-45db-a585-9c7cd632ebee admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.195431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3904e5-deca-45db-a585-9c7cd632ebee admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.196074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3904e5-deca-45db-a585-9c7cd632ebee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:07.198653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3904e5-deca-45db-a585-9c7cd632ebee admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:07.225796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1329/2642] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:07 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:55:07.226874 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3904e5-deca-45db-a585-9c7cd632ebee admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:07.228915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5e5aa-2e5b-466e-a0ec-cd51ae09a0d9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:07.233345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cd9c2-55bb-46c9-8d45-3a7b8b8a9ac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:07.234152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cd9c2-55bb-46c9-8d45-3a7b8b8a9ac0 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.234530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cd9c2-55bb-46c9-8d45-3a7b8b8a9ac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.234891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cd9c2-55bb-46c9-8d45-3a7b8b8a9ac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.237284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3904e5-deca-45db-a585-9c7cd632ebee admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:07.242458 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747fe5a-b1b9-4878-873f-b4b2d485ee47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:07.243159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747fe5a-b1b9-4878-873f-b4b2d485ee47 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.243504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747fe5a-b1b9-4878-873f-b4b2d485ee47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.243786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747fe5a-b1b9-4878-873f-b4b2d485ee47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.244431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747fe5a-b1b9-4878-873f-b4b2d485ee47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:07.244963 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8cd9c2-55bb-46c9-8d45-3a7b8b8a9ac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:07.246100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747fe5a-b1b9-4878-873f-b4b2d485ee47 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:07.246390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8cd9c2-55bb-46c9-8d45-3a7b8b8a9ac0 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:07.255914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1314/2643] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:55:07.256906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1330/2644] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:55:07.261136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cd9c2-55bb-46c9-8d45-3a7b8b8a9ac0 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:07.270619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cd9c2-55bb-46c9-8d45-3a7b8b8a9ac0 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:07.275784 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe99ab-df86-415e-9fd9-a587eaff7db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:07.276427 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe99ab-df86-415e-9fd9-a587eaff7db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.276741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe99ab-df86-415e-9fd9-a587eaff7db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.277015 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe99ab-df86-415e-9fd9-a587eaff7db5 admin admin] PATH_INFO: `/v3/projects/9a4b9d5aa0444b8c846683c2cfc8005d/users/ea3a26a38f944623a832c575c042a14c/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.295495 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe99ab-df86-415e-9fd9-a587eaff7db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a4b9d5aa0444b8c846683c2cfc8005d, user_id=ea3a26a38f944623a832c575c042a14c, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:07.298288 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe99ab-df86-415e-9fd9-a587eaff7db5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:07.313383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1315/2645] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:07 2022] PUT /identity/v3/projects/9a4b9d5aa0444b8c846683c2cfc8005d/users/ea3a26a38f944623a832c575c042a14c/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:55:07.317908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747fe5a-b1b9-4878-873f-b4b2d485ee47 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:07.329366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747fe5a-b1b9-4878-873f-b4b2d485ee47 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:07.391113 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96828812-024a-4f55-a5ac-94e7f38b47a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:07.391789 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96828812-024a-4f55-a5ac-94e7f38b47a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.392345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96828812-024a-4f55-a5ac-94e7f38b47a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.392345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96828812-024a-4f55-a5ac-94e7f38b47a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.392981 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96828812-024a-4f55-a5ac-94e7f38b47a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:07.394741 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96828812-024a-4f55-a5ac-94e7f38b47a9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:07.413647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1331/2646] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:07.418268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe99ab-df86-415e-9fd9-a587eaff7db5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:07.429204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe99ab-df86-415e-9fd9-a587eaff7db5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:07.434936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb9a54-0fc5-4963-888c-202b78a93056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:07.435664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb9a54-0fc5-4963-888c-202b78a93056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.436013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb9a54-0fc5-4963-888c-202b78a93056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.436353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb9a54-0fc5-4963-888c-202b78a93056 admin admin] PATH_INFO: `/v3/projects/9a4b9d5aa0444b8c846683c2cfc8005d/users/ea3a26a38f944623a832c575c042a14c/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.438388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb9a54-0fc5-4963-888c-202b78a93056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a4b9d5aa0444b8c846683c2cfc8005d, user_id=ea3a26a38f944623a832c575c042a14c, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:07.441561 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb9a54-0fc5-4963-888c-202b78a93056 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:07.453863 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1316/2647] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:07 2022] PUT /identity/v3/projects/9a4b9d5aa0444b8c846683c2cfc8005d/users/ea3a26a38f944623a832c575c042a14c/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:55:07.465490 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c61b3-6128-401c-98c4-85be6c44d769 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.465773 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c61b3-6128-401c-98c4-85be6c44d769 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.466049 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c61b3-6128-401c-98c4-85be6c44d769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.494085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d14c61b3-6128-401c-98c4-85be6c44d769 None None] MFA Rules not processed for user `ea3a26a38f944623a832c575c042a14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:07.495330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d14c61b3-6128-401c-98c4-85be6c44d769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:07.544077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb9a54-0fc5-4963-888c-202b78a93056 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:07.545866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1beb9a54-0fc5-4963-888c-202b78a93056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:07.590691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1332/2648] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:07.603820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c61b3-6128-401c-98c4-85be6c44d769 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:07.605356 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d14c61b3-6128-401c-98c4-85be6c44d769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:07.607473 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:07.642824 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb9a54-0fc5-4963-888c-202b78a93056 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:07.648933 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:07.649677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.650102 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.650530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] PATH_INFO: `/v3/projects/ca915d45a4934088a389750b6c923926` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.651923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:07.652139 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca915d45a4934088a389750b6c923926)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:07.655095 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:07.667131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c61b3-6128-401c-98c4-85be6c44d769 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:07.670818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc70dae-d37f-408b-b7cb-9cba20cd0452 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:07.671427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc70dae-d37f-408b-b7cb-9cba20cd0452 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.671707 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc70dae-d37f-408b-b7cb-9cba20cd0452 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.671977 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc70dae-d37f-408b-b7cb-9cba20cd0452 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.683203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc70dae-d37f-408b-b7cb-9cba20cd0452 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:07.685181 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc70dae-d37f-408b-b7cb-9cba20cd0452 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:07.697290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1333/2649] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:07.706464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca915d45a4934088a389750b6c923926', 'request_id': 'req-84a69459-f095-4cc7-956e-9e1bcb0b772a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.712983 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca915d45a4934088a389750b6c923926', 'request_id': 'req-84a69459-f095-4cc7-956e-9e1bcb0b772a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.719522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca915d45a4934088a389750b6c923926', 'request_id': 'req-84a69459-f095-4cc7-956e-9e1bcb0b772a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.735233 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] The token cache is being invalidate because project ca915d45a4934088a389750b6c923926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:07.735539 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.741419 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.743235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1317/2650] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:07 2022] DELETE /identity/v3/projects/ca915d45a4934088a389750b6c923926 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:55:07.747877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc70dae-d37f-408b-b7cb-9cba20cd0452 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:07.766801 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc70dae-d37f-408b-b7cb-9cba20cd0452 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:07.836637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53599924-3979-4c72-995d-fa65a88c624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:07.837275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53599924-3979-4c72-995d-fa65a88c624f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:07.837568 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53599924-3979-4c72-995d-fa65a88c624f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:07.837840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53599924-3979-4c72-995d-fa65a88c624f admin admin] PATH_INFO: `/v3/users/0d7e984fd774456f958c4100cee9e3c6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:07.838989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53599924-3979-4c72-995d-fa65a88c624f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7e984fd774456f958c4100cee9e3c6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:07.840963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53599924-3979-4c72-995d-fa65a88c624f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:07.899375 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-53599924-3979-4c72-995d-fa65a88c624f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7e984fd774456f958c4100cee9e3c6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.909790 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-53599924-3979-4c72-995d-fa65a88c624f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7e984fd774456f958c4100cee9e3c6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.910496 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-53599924-3979-4c72-995d-fa65a88c624f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7e984fd774456f958c4100cee9e3c6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.911150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-53599924-3979-4c72-995d-fa65a88c624f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7e984fd774456f958c4100cee9e3c6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.919184 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-53599924-3979-4c72-995d-fa65a88c624f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7e984fd774456f958c4100cee9e3c6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:07.938142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1334/2651] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:07 2022] DELETE /identity/v3/users/0d7e984fd774456f958c4100cee9e3c6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:08.267683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:08.269226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:08.283430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69459-f095-4cc7-956e-9e1bcb0b772a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:08.352933 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:08.353584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:08.353870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:08.354148 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] PATH_INFO: `/v3/projects/583e18fadca54c839368e74efa8b5f16` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:08.355285 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=583e18fadca54c839368e74efa8b5f16)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:08.358150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:08.416467 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583e18fadca54c839368e74efa8b5f16', 'request_id': 'req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:08.423082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583e18fadca54c839368e74efa8b5f16', 'request_id': 'req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:08.430793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583e18fadca54c839368e74efa8b5f16', 'request_id': 'req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:08.433193 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:08.449282 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] The token cache is being invalidate because project 583e18fadca54c839368e74efa8b5f16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:08.449730 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:08.450212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b507b0ae-6bd9-4677-b497-5a3bd8bc9f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:08.451692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1318/2652] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:08 2022] DELETE /identity/v3/projects/583e18fadca54c839368e74efa8b5f16 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:08.457353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17121ab1-0309-4da7-8b03-638c283840ae None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:08.457894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17121ab1-0309-4da7-8b03-638c283840ae None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:08.457894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17121ab1-0309-4da7-8b03-638c283840ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:08.479343 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-17121ab1-0309-4da7-8b03-638c283840ae None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:08.480601 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-17121ab1-0309-4da7-8b03-638c283840ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:08.545894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1335/2653] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:08.554376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8f3b8-a662-41ca-9709-96ba13182a19 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:08.554942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8f3b8-a662-41ca-9709-96ba13182a19 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:08.555189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8f3b8-a662-41ca-9709-96ba13182a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:08.577336 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4fc8f3b8-a662-41ca-9709-96ba13182a19 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:08.578544 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4fc8f3b8-a662-41ca-9709-96ba13182a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:08.591077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1319/2654] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:08.595373 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17121ab1-0309-4da7-8b03-638c283840ae None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:08.606231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:08.613873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17121ab1-0309-4da7-8b03-638c283840ae None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:08.623222 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df9e6c-14ab-4c3a-8795-eb49051ee4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:08.623843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df9e6c-14ab-4c3a-8795-eb49051ee4cd admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:08.624121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df9e6c-14ab-4c3a-8795-eb49051ee4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:08.624397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df9e6c-14ab-4c3a-8795-eb49051ee4cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:08.625107 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df9e6c-14ab-4c3a-8795-eb49051ee4cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:55:08.625413 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df9e6c-14ab-4c3a-8795-eb49051ee4cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:08.627284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df9e6c-14ab-4c3a-8795-eb49051ee4cd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:08.642595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1336/2655] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:55:08.647673 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8f3b8-a662-41ca-9709-96ba13182a19 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:08.656973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8f3b8-a662-41ca-9709-96ba13182a19 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:08.661968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca19cf-93cd-4752-a61f-488df4a5324b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:08.662973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca19cf-93cd-4752-a61f-488df4a5324b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:08.663255 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca19cf-93cd-4752-a61f-488df4a5324b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:08.663530 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca19cf-93cd-4752-a61f-488df4a5324b admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:08.664164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca19cf-93cd-4752-a61f-488df4a5324b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:08.666115 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca19cf-93cd-4752-a61f-488df4a5324b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:08.716524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1320/2656] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:08 2022] POST /identity/v3/projects => generated 409 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:08.719399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df9e6c-14ab-4c3a-8795-eb49051ee4cd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:08.730437 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df9e6c-14ab-4c3a-8795-eb49051ee4cd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:08.800210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9449d19-2296-4950-98bc-ddd1a2502622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:08.800943 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9449d19-2296-4950-98bc-ddd1a2502622 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:08.801287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9449d19-2296-4950-98bc-ddd1a2502622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:08.801613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9449d19-2296-4950-98bc-ddd1a2502622 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:08.802840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9449d19-2296-4950-98bc-ddd1a2502622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:08.805849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9449d19-2296-4950-98bc-ddd1a2502622 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:08.836975 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1337/2657] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:08 2022] POST /identity/v3/users => generated 305 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:08.837700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca19cf-93cd-4752-a61f-488df4a5324b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:08.849734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca19cf-93cd-4752-a61f-488df4a5324b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:08.855680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1ceb8-f259-4014-8937-f44b76f2fcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:08.856452 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1ceb8-f259-4014-8937-f44b76f2fcaa admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:08.856824 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1ceb8-f259-4014-8937-f44b76f2fcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:08.857425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1ceb8-f259-4014-8937-f44b76f2fcaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:08.858300 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1ceb8-f259-4014-8937-f44b76f2fcaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:08.860341 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1ceb8-f259-4014-8937-f44b76f2fcaa admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:08.879787 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1321/2658] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:55:08.886117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9449d19-2296-4950-98bc-ddd1a2502622 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:08.898961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9449d19-2296-4950-98bc-ddd1a2502622 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:08.903911 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b07cb-32a7-408f-9985-65042c551083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:08.904512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b07cb-32a7-408f-9985-65042c551083 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:08.904814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b07cb-32a7-408f-9985-65042c551083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:08.905099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b07cb-32a7-408f-9985-65042c551083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:08.905740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b07cb-32a7-408f-9985-65042c551083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:08.907333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b07cb-32a7-408f-9985-65042c551083 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:08.922460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1338/2659] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:55:08.924083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1ceb8-f259-4014-8937-f44b76f2fcaa admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:08.937272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1ceb8-f259-4014-8937-f44b76f2fcaa admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:08.942890 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c89390-b46c-4495-82fb-6e003efa0425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:08.943254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c89390-b46c-4495-82fb-6e003efa0425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:08.943520 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c89390-b46c-4495-82fb-6e003efa0425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:08.943796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c89390-b46c-4495-82fb-6e003efa0425 admin admin] PATH_INFO: `/v3/projects/61aff2beae19496ebe0e94ef70bcd350/users/ab84fb2aa8d54a90b34d5ad239df96dc/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:08.960777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c89390-b46c-4495-82fb-6e003efa0425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61aff2beae19496ebe0e94ef70bcd350, user_id=ab84fb2aa8d54a90b34d5ad239df96dc, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:08.963340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c89390-b46c-4495-82fb-6e003efa0425 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:08.977808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1322/2660] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:08 2022] PUT /identity/v3/projects/61aff2beae19496ebe0e94ef70bcd350/users/ab84fb2aa8d54a90b34d5ad239df96dc/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:08.997937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61df5d3-b7e0-4999-95c1-c868c934e227 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:08.998277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61df5d3-b7e0-4999-95c1-c868c934e227 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:08.998794 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61df5d3-b7e0-4999-95c1-c868c934e227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.042166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d61df5d3-b7e0-4999-95c1-c868c934e227 None None] MFA Rules not processed for user `ab84fb2aa8d54a90b34d5ad239df96dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:09.043483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d61df5d3-b7e0-4999-95c1-c868c934e227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:09.124939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1339/2661] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:08 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:09.130392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c89390-b46c-4495-82fb-6e003efa0425 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:09.220305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c89390-b46c-4495-82fb-6e003efa0425 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:09.227629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f98638-63b9-42d3-bb03-a5f40505d6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:09.228264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f98638-63b9-42d3-bb03-a5f40505d6e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:09.228666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f98638-63b9-42d3-bb03-a5f40505d6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:09.228955 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f98638-63b9-42d3-bb03-a5f40505d6e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.229621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f98638-63b9-42d3-bb03-a5f40505d6e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:09.231827 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f98638-63b9-42d3-bb03-a5f40505d6e4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:09.267413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1323/2662] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:09 2022] POST /identity/v3/projects => generated 407 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:09.268807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61df5d3-b7e0-4999-95c1-c868c934e227 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:09.271287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f98638-63b9-42d3-bb03-a5f40505d6e4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:09.273759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f9f98638-63b9-42d3-bb03-a5f40505d6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:09.281642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61df5d3-b7e0-4999-95c1-c868c934e227 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:09.341067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:09.350969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7fc21-6f2f-4ce9-b3b6-4f3575c461e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:09.351803 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7fc21-6f2f-4ce9-b3b6-4f3575c461e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:09.352206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7fc21-6f2f-4ce9-b3b6-4f3575c461e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:09.352955 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7fc21-6f2f-4ce9-b3b6-4f3575c461e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.353786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7fc21-6f2f-4ce9-b3b6-4f3575c461e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:09.356166 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7fc21-6f2f-4ce9-b3b6-4f3575c461e5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:09.359546 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f98638-63b9-42d3-bb03-a5f40505d6e4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:09.364675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9845309-7122-4a16-9f25-e9dbd7380727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:09.365312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9845309-7122-4a16-9f25-e9dbd7380727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:09.366600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9845309-7122-4a16-9f25-e9dbd7380727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:09.366911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9845309-7122-4a16-9f25-e9dbd7380727 admin admin] PATH_INFO: `/v3/users/9793bff878db44f1923824b297e2285c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.368367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9845309-7122-4a16-9f25-e9dbd7380727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9793bff878db44f1923824b297e2285c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:09.370385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9845309-7122-4a16-9f25-e9dbd7380727 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:09.381379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1340/2663] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:09 2022] POST /identity/v3/users => generated 310 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:55:09.387621 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7fc21-6f2f-4ce9-b3b6-4f3575c461e5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:09.389672 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:INSERT=4,keystone:DELETE=2,keystone:SELECT=94 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:09.402510 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7fc21-6f2f-4ce9-b3b6-4f3575c461e5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:09.409748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecd123-9a23-4479-856b-1e7b501d890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:09.410740 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd123-9a23-4479-856b-1e7b501d890f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:09.411191 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd123-9a23-4479-856b-1e7b501d890f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:09.411602 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd123-9a23-4479-856b-1e7b501d890f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.413329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ecd123-9a23-4479-856b-1e7b501d890f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:09.415604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ecd123-9a23-4479-856b-1e7b501d890f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:09.429092 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1341/2664] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:55:09.433757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecd123-9a23-4479-856b-1e7b501d890f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:09.438712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f9845309-7122-4a16-9f25-e9dbd7380727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9793bff878db44f1923824b297e2285c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:09.439586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f9845309-7122-4a16-9f25-e9dbd7380727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9793bff878db44f1923824b297e2285c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:09.445616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecd123-9a23-4479-856b-1e7b501d890f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:09.448644 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f9845309-7122-4a16-9f25-e9dbd7380727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9793bff878db44f1923824b297e2285c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:09.451637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0332c1-7aeb-4064-8e27-02f73c57e57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:09.452298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0332c1-7aeb-4064-8e27-02f73c57e57e admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:09.452671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0332c1-7aeb-4064-8e27-02f73c57e57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:09.453015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0332c1-7aeb-4064-8e27-02f73c57e57e admin admin] PATH_INFO: `/v3/projects/1bf2da7494554069b7edc1702c78bf3d/users/0bb589c3916a48b79a134228f8fcaee9/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.457706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f9845309-7122-4a16-9f25-e9dbd7380727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9793bff878db44f1923824b297e2285c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:09.458558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f9845309-7122-4a16-9f25-e9dbd7380727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9793bff878db44f1923824b297e2285c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:09.470898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0332c1-7aeb-4064-8e27-02f73c57e57e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf2da7494554069b7edc1702c78bf3d, user_id=0bb589c3916a48b79a134228f8fcaee9, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:09.476854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0332c1-7aeb-4064-8e27-02f73c57e57e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:09.482204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1324/2665] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:09 2022] DELETE /identity/v3/users/9793bff878db44f1923824b297e2285c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:09.492581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f7d7e-8654-4a42-8843-760f6058efa4 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:09.493064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f7d7e-8654-4a42-8843-760f6058efa4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:09.493503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f7d7e-8654-4a42-8843-760f6058efa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.495433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1342/2666] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:09 2022] PUT /identity/v3/projects/1bf2da7494554069b7edc1702c78bf3d/users/0bb589c3916a48b79a134228f8fcaee9/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:55:09.499722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0332c1-7aeb-4064-8e27-02f73c57e57e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:09.501766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0f0332c1-7aeb-4064-8e27-02f73c57e57e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:09.519887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-fb1f7d7e-8654-4a42-8843-760f6058efa4 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:09.521118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fb1f7d7e-8654-4a42-8843-760f6058efa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:09.572202 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0332c1-7aeb-4064-8e27-02f73c57e57e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:09.577112 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99678765-9701-4b32-9343-952d63f346fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:09.577717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99678765-9701-4b32-9343-952d63f346fe admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:09.577999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99678765-9701-4b32-9343-952d63f346fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:09.578273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99678765-9701-4b32-9343-952d63f346fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.578914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99678765-9701-4b32-9343-952d63f346fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:09.580524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99678765-9701-4b32-9343-952d63f346fe admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:09.588085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1325/2667] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:09.591472 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1343/2668] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:09.596390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f7d7e-8654-4a42-8843-760f6058efa4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:09.605503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99678765-9701-4b32-9343-952d63f346fe admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:09.608432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-99678765-9701-4b32-9343-952d63f346fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:09.614710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f7d7e-8654-4a42-8843-760f6058efa4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:09.623116 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44496a15-26aa-4ea8-8d0f-28f88f7dbdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:09.624205 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44496a15-26aa-4ea8-8d0f-28f88f7dbdd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:09.624757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44496a15-26aa-4ea8-8d0f-28f88f7dbdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:09.625239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44496a15-26aa-4ea8-8d0f-28f88f7dbdd3 admin admin] PATH_INFO: `/v3/projects/1bf2da7494554069b7edc1702c78bf3d/users/0bb589c3916a48b79a134228f8fcaee9/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.628513 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44496a15-26aa-4ea8-8d0f-28f88f7dbdd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf2da7494554069b7edc1702c78bf3d, user_id=0bb589c3916a48b79a134228f8fcaee9, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:09.633502 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44496a15-26aa-4ea8-8d0f-28f88f7dbdd3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:09.653893 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1326/2669] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:09 2022] PUT /identity/v3/projects/1bf2da7494554069b7edc1702c78bf3d/users/0bb589c3916a48b79a134228f8fcaee9/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:09.666221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7db68-4c78-4b68-b5e2-754589194ca1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:09.666894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7db68-4c78-4b68-b5e2-754589194ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:09.667708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7db68-4c78-4b68-b5e2-754589194ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.676163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99678765-9701-4b32-9343-952d63f346fe admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:09.714937 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-46b7db68-4c78-4b68-b5e2-754589194ca1 None None] MFA Rules not processed for user `0bb589c3916a48b79a134228f8fcaee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:09.716177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-46b7db68-4c78-4b68-b5e2-754589194ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:09.750080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918c70d-eaea-40d4-92fa-69e38e8c4449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:09.750600 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918c70d-eaea-40d4-92fa-69e38e8c4449 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:09.750882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918c70d-eaea-40d4-92fa-69e38e8c4449 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:09.751159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918c70d-eaea-40d4-92fa-69e38e8c4449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.769370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:09.796330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1327/2670] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:09 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:09.813451 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7db68-4c78-4b68-b5e2-754589194ca1 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:09.815892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-46b7db68-4c78-4b68-b5e2-754589194ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:09.826621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:09.839964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918c70d-eaea-40d4-92fa-69e38e8c4449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:09.841826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918c70d-eaea-40d4-92fa-69e38e8c4449 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:09.853691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1344/2671] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:55:09.890576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7db68-4c78-4b68-b5e2-754589194ca1 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:09.895634 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1de204-e803-48a9-9be9-261ed955e86c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:09.896272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1de204-e803-48a9-9be9-261ed955e86c service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:09.896654 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1de204-e803-48a9-9be9-261ed955e86c service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:09.897121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1de204-e803-48a9-9be9-261ed955e86c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:09.909256 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1de204-e803-48a9-9be9-261ed955e86c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:09.911492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1de204-e803-48a9-9be9-261ed955e86c service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:09.928003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1328/2672] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:09 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:55:10.143105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918c70d-eaea-40d4-92fa-69e38e8c4449 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:10.154677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918c70d-eaea-40d4-92fa-69e38e8c4449 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:10.158447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29a69d-4498-4926-a124-6c5df39287a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:10.159362 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29a69d-4498-4926-a124-6c5df39287a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:10.159362 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29a69d-4498-4926-a124-6c5df39287a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:10.159638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29a69d-4498-4926-a124-6c5df39287a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:10.161347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ee29a69d-4498-4926-a124-6c5df39287a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:10.213398 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1de204-e803-48a9-9be9-261ed955e86c service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:10.216396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2a1de204-e803-48a9-9be9-261ed955e86c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:10.234414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1de204-e803-48a9-9be9-261ed955e86c service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:10.239706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:10.242092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:10.242092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:10.242092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] PATH_INFO: `/v3/projects/6b8afeb4e04a44ec858ce5404fd18757` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:10.242770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b8afeb4e04a44ec858ce5404fd18757)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:10.246013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:10.250751 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29a69d-4498-4926-a124-6c5df39287a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:10.252248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29a69d-4498-4926-a124-6c5df39287a6 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:10.263389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1345/2673] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:10 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:55:10.320109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b8afeb4e04a44ec858ce5404fd18757', 'request_id': 'req-765878d1-3c9e-41c6-9707-68019c5c663f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:10.326844 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b8afeb4e04a44ec858ce5404fd18757', 'request_id': 'req-765878d1-3c9e-41c6-9707-68019c5c663f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:10.336157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b8afeb4e04a44ec858ce5404fd18757', 'request_id': 'req-765878d1-3c9e-41c6-9707-68019c5c663f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:10.353726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] The token cache is being invalidate because project 6b8afeb4e04a44ec858ce5404fd18757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:10.354013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:10.354650 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:10.356409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1329/2674] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:10 2022] DELETE /identity/v3/projects/6b8afeb4e04a44ec858ce5404fd18757 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:10.360176 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29a69d-4498-4926-a124-6c5df39287a6 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:10.371410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29a69d-4498-4926-a124-6c5df39287a6 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:10.448426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d84bd-e0a8-4df4-a663-055ef02b31a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:10.449292 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d84bd-e0a8-4df4-a663-055ef02b31a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:10.449672 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d84bd-e0a8-4df4-a663-055ef02b31a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:10.450303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d84bd-e0a8-4df4-a663-055ef02b31a6 admin admin] PATH_INFO: `/v3/users/ea3a26a38f944623a832c575c042a14c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:10.451792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d84bd-e0a8-4df4-a663-055ef02b31a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea3a26a38f944623a832c575c042a14c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:10.453919 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d84bd-e0a8-4df4-a663-055ef02b31a6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:10.519533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-925d84bd-e0a8-4df4-a663-055ef02b31a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea3a26a38f944623a832c575c042a14c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:10.526442 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:10.530393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-925d84bd-e0a8-4df4-a663-055ef02b31a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea3a26a38f944623a832c575c042a14c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:10.531761 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-925d84bd-e0a8-4df4-a663-055ef02b31a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea3a26a38f944623a832c575c042a14c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:10.533096 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-925d84bd-e0a8-4df4-a663-055ef02b31a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea3a26a38f944623a832c575c042a14c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:10.545459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-925d84bd-e0a8-4df4-a663-055ef02b31a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea3a26a38f944623a832c575c042a14c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:10.566013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1346/2675] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:10 2022] DELETE /identity/v3/users/ea3a26a38f944623a832c575c042a14c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:10.879179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:10.880697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:10.894998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765878d1-3c9e-41c6-9707-68019c5c663f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:10.957443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:10.958074 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:10.958353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:10.958628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] PATH_INFO: `/v3/projects/9a4b9d5aa0444b8c846683c2cfc8005d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:10.960422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a4b9d5aa0444b8c846683c2cfc8005d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:10.962347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:11.008360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:11.026003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a4b9d5aa0444b8c846683c2cfc8005d', 'request_id': 'req-4b096c47-ea46-4d95-a17b-94e1ae0055cd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:11.033096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a4b9d5aa0444b8c846683c2cfc8005d', 'request_id': 'req-4b096c47-ea46-4d95-a17b-94e1ae0055cd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:11.039902 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a4b9d5aa0444b8c846683c2cfc8005d', 'request_id': 'req-4b096c47-ea46-4d95-a17b-94e1ae0055cd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:11.053291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] The token cache is being invalidate because project 9a4b9d5aa0444b8c846683c2cfc8005d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:11.053674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:11.054337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-4b096c47-ea46-4d95-a17b-94e1ae0055cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:11.055837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1330/2676] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:10 2022] DELETE /identity/v3/projects/9a4b9d5aa0444b8c846683c2cfc8005d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:55:11.060791 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eca75e-0afe-4a71-be54-b1daf8535599 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:11.061076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eca75e-0afe-4a71-be54-b1daf8535599 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:11.061345 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eca75e-0afe-4a71-be54-b1daf8535599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:11.082251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a1eca75e-0afe-4a71-be54-b1daf8535599 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:11.083359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a1eca75e-0afe-4a71-be54-b1daf8535599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:11.148031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1347/2677] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:11.155932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6f683-f499-4359-bedc-9ef6e7568bb8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:11.156242 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6f683-f499-4359-bedc-9ef6e7568bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:11.156555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6f683-f499-4359-bedc-9ef6e7568bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:11.176953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b9c6f683-f499-4359-bedc-9ef6e7568bb8 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:11.178464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b9c6f683-f499-4359-bedc-9ef6e7568bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:11.187219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1331/2678] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:11.192374 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eca75e-0afe-4a71-be54-b1daf8535599 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:11.207951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eca75e-0afe-4a71-be54-b1daf8535599 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:11.217284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfed264-01db-43c5-b173-809020a18078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:11.218382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfed264-01db-43c5-b173-809020a18078 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:11.218928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfed264-01db-43c5-b173-809020a18078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:11.219450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfed264-01db-43c5-b173-809020a18078 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:11.220754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfed264-01db-43c5-b173-809020a18078 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:55:11.221362 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfed264-01db-43c5-b173-809020a18078 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:11.225329 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfed264-01db-43c5-b173-809020a18078 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:11.240776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1348/2679] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:55:11.245459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6f683-f499-4359-bedc-9ef6e7568bb8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:11.254688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6f683-f499-4359-bedc-9ef6e7568bb8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:11.259378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde67ce-ddac-479e-a873-2c2872365ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:11.259968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde67ce-ddac-479e-a873-2c2872365ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:11.260248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde67ce-ddac-479e-a873-2c2872365ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:11.260547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde67ce-ddac-479e-a873-2c2872365ba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:11.261174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde67ce-ddac-479e-a873-2c2872365ba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:11.263058 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde67ce-ddac-479e-a873-2c2872365ba8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:11.294482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1332/2680] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:11 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:11.299274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfed264-01db-43c5-b173-809020a18078 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:11.312367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfed264-01db-43c5-b173-809020a18078 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:11.380007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3623ccb-a66d-458c-8b07-388c9f7b84df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:11.380543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3623ccb-a66d-458c-8b07-388c9f7b84df admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:11.380830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3623ccb-a66d-458c-8b07-388c9f7b84df admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:11.382177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3623ccb-a66d-458c-8b07-388c9f7b84df admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:11.382841 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3623ccb-a66d-458c-8b07-388c9f7b84df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:11.384795 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3623ccb-a66d-458c-8b07-388c9f7b84df admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:11.411207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1349/2681] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:11 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:11.416058 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde67ce-ddac-479e-a873-2c2872365ba8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:11.427170 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde67ce-ddac-479e-a873-2c2872365ba8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:11.432287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0c37a-67fa-4323-adf9-d10ae462326c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:11.432932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c37a-67fa-4323-adf9-d10ae462326c admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:11.433221 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c37a-67fa-4323-adf9-d10ae462326c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:11.433500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c37a-67fa-4323-adf9-d10ae462326c admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:11.434156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0c37a-67fa-4323-adf9-d10ae462326c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:11.435760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0c37a-67fa-4323-adf9-d10ae462326c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:11.447089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1333/2682] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:11.451357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3623ccb-a66d-458c-8b07-388c9f7b84df admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:11.462112 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3623ccb-a66d-458c-8b07-388c9f7b84df admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:11.466812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de6152-4764-4b2a-809d-91666b7fc022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:11.467403 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de6152-4764-4b2a-809d-91666b7fc022 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:11.467682 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de6152-4764-4b2a-809d-91666b7fc022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:11.467949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de6152-4764-4b2a-809d-91666b7fc022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:11.468627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de6152-4764-4b2a-809d-91666b7fc022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:11.470224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de6152-4764-4b2a-809d-91666b7fc022 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:11.480304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1350/2683] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:11.484461 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0c37a-67fa-4323-adf9-d10ae462326c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:11.495904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0c37a-67fa-4323-adf9-d10ae462326c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:11.501341 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfec9a2-3c1d-43d1-b06b-8253102a1f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:11.502026 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec9a2-3c1d-43d1-b06b-8253102a1f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:11.502313 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec9a2-3c1d-43d1-b06b-8253102a1f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:11.502595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec9a2-3c1d-43d1-b06b-8253102a1f89 admin admin] PATH_INFO: `/v3/projects/7ea465e2cddc4badb834b3fd45ea14cc/users/bc56244c98bd4dbd896cb7f3911b4502/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:11.522234 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfec9a2-3c1d-43d1-b06b-8253102a1f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea465e2cddc4badb834b3fd45ea14cc, user_id=bc56244c98bd4dbd896cb7f3911b4502, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:11.525149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfec9a2-3c1d-43d1-b06b-8253102a1f89 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:11.539052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1334/2684] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:11 2022] PUT /identity/v3/projects/7ea465e2cddc4badb834b3fd45ea14cc/users/bc56244c98bd4dbd896cb7f3911b4502/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:11.551047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3099f-1121-4084-9d32-2eaafa880376 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:11.551595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3099f-1121-4084-9d32-2eaafa880376 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:11.551720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3099f-1121-4084-9d32-2eaafa880376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:11.580247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d4e3099f-1121-4084-9d32-2eaafa880376 None None] MFA Rules not processed for user `bc56244c98bd4dbd896cb7f3911b4502`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:11.581474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d4e3099f-1121-4084-9d32-2eaafa880376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:11.617208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:11.674978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1351/2685] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:11.684008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfec9a2-3c1d-43d1-b06b-8253102a1f89 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:11.763426 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfec9a2-3c1d-43d1-b06b-8253102a1f89 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:11.768395 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e8faa-ae2e-4852-9e36-c6719da730ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:11.769054 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e8faa-ae2e-4852-9e36-c6719da730ec admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:11.769339 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e8faa-ae2e-4852-9e36-c6719da730ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:11.769618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e8faa-ae2e-4852-9e36-c6719da730ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:11.770265 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e8faa-ae2e-4852-9e36-c6719da730ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:11.772370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e8faa-ae2e-4852-9e36-c6719da730ec admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:11.827818 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1335/2686] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:11 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:11.829138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3099f-1121-4084-9d32-2eaafa880376 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:11.847404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3099f-1121-4084-9d32-2eaafa880376 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:11.978053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c5ac5-5c26-42b0-b75a-eea440cd04e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:11.978749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c5ac5-5c26-42b0-b75a-eea440cd04e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:11.979085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c5ac5-5c26-42b0-b75a-eea440cd04e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:11.979528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c5ac5-5c26-42b0-b75a-eea440cd04e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:11.980288 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c5ac5-5c26-42b0-b75a-eea440cd04e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:11.983014 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c5ac5-5c26-42b0-b75a-eea440cd04e7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:12.010598 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1352/2687] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:11 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:12.015632 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e8faa-ae2e-4852-9e36-c6719da730ec admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:12.026925 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e8faa-ae2e-4852-9e36-c6719da730ec admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:12.031710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbbe14-3849-4959-9c59-008c1705e9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:12.032333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbbe14-3849-4959-9c59-008c1705e9af admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:12.032655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbbe14-3849-4959-9c59-008c1705e9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:12.032933 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbbe14-3849-4959-9c59-008c1705e9af admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:12.033589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbbe14-3849-4959-9c59-008c1705e9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:12.035205 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbbe14-3849-4959-9c59-008c1705e9af admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:12.050738 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1336/2688] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:55:12.051997 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c5ac5-5c26-42b0-b75a-eea440cd04e7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:12.064050 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c5ac5-5c26-42b0-b75a-eea440cd04e7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:12.069736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f407d0-df26-46c0-a418-361a6235dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:12.070616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f407d0-df26-46c0-a418-361a6235dc04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:12.070892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f407d0-df26-46c0-a418-361a6235dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:12.071163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f407d0-df26-46c0-a418-361a6235dc04 admin admin] PATH_INFO: `/v3/projects/0e8a1f33bbc74062b1232a8986386a6c/users/81051acf9c2c4719911f2725e40e70e5/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:12.088699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f407d0-df26-46c0-a418-361a6235dc04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8a1f33bbc74062b1232a8986386a6c, user_id=81051acf9c2c4719911f2725e40e70e5, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:12.091393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f407d0-df26-46c0-a418-361a6235dc04 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:12.105954 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1353/2689] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:12 2022] PUT /identity/v3/projects/0e8a1f33bbc74062b1232a8986386a6c/users/81051acf9c2c4719911f2725e40e70e5/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:12.110107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbbe14-3849-4959-9c59-008c1705e9af admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:12.120661 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbbe14-3849-4959-9c59-008c1705e9af admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:12.154776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:12.185333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e6a5b-b7e4-4573-809d-9aea3d6077e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:12.185953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e6a5b-b7e4-4573-809d-9aea3d6077e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:12.186320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e6a5b-b7e4-4573-809d-9aea3d6077e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:12.186497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e6a5b-b7e4-4573-809d-9aea3d6077e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:12.187147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e6a5b-b7e4-4573-809d-9aea3d6077e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:12.188790 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e6a5b-b7e4-4573-809d-9aea3d6077e3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:12.200413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1337/2690] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:12.204718 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f407d0-df26-46c0-a418-361a6235dc04 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:12.218492 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f407d0-df26-46c0-a418-361a6235dc04 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:12.223646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26aab39-6838-4d66-8e6c-696d44d0e8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:12.224316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26aab39-6838-4d66-8e6c-696d44d0e8af admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:12.224762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26aab39-6838-4d66-8e6c-696d44d0e8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:12.225154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26aab39-6838-4d66-8e6c-696d44d0e8af admin admin] PATH_INFO: `/v3/projects/0e8a1f33bbc74062b1232a8986386a6c/users/81051acf9c2c4719911f2725e40e70e5/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:12.227168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26aab39-6838-4d66-8e6c-696d44d0e8af admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8a1f33bbc74062b1232a8986386a6c, user_id=81051acf9c2c4719911f2725e40e70e5, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:12.229971 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26aab39-6838-4d66-8e6c-696d44d0e8af admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:12.241315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1354/2691] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:12 2022] PUT /identity/v3/projects/0e8a1f33bbc74062b1232a8986386a6c/users/81051acf9c2c4719911f2725e40e70e5/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:12.252059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f8a93-d4ef-4c71-b54a-556873b475dd None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:12.252484 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f8a93-d4ef-4c71-b54a-556873b475dd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:12.252900 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f8a93-d4ef-4c71-b54a-556873b475dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:12.288564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ac6f8a93-d4ef-4c71-b54a-556873b475dd None None] MFA Rules not processed for user `81051acf9c2c4719911f2725e40e70e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:12.291250 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ac6f8a93-d4ef-4c71-b54a-556873b475dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:12.382325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1338/2692] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:12.394730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26aab39-6838-4d66-8e6c-696d44d0e8af admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:12.396531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f26aab39-6838-4d66-8e6c-696d44d0e8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:12.466382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26aab39-6838-4d66-8e6c-696d44d0e8af admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:12.470037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2515f5-c3bc-4c83-be98-064fbdd018f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:12.470638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2515f5-c3bc-4c83-be98-064fbdd018f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:12.470934 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2515f5-c3bc-4c83-be98-064fbdd018f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:12.471207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2515f5-c3bc-4c83-be98-064fbdd018f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:12.505346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:12.533208 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2515f5-c3bc-4c83-be98-064fbdd018f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:12.534651 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2515f5-c3bc-4c83-be98-064fbdd018f2 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:12.546220 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1355/2693] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:22.378447 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=49 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:22.465932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f8a93-d4ef-4c71-b54a-556873b475dd None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:22.476392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f8a93-d4ef-4c71-b54a-556873b475dd None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:22.481639 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b5056-8af7-4ef6-b73f-513075c52046 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:22.482239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b5056-8af7-4ef6-b73f-513075c52046 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:22.482527 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b5056-8af7-4ef6-b73f-513075c52046 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:22.482808 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b5056-8af7-4ef6-b73f-513075c52046 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:22.492745 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b5056-8af7-4ef6-b73f-513075c52046 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:22.494276 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b5056-8af7-4ef6-b73f-513075c52046 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:22.503601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1339/2694] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:22.541725 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=12 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:24.046040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2515f5-c3bc-4c83-be98-064fbdd018f2 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:24.047435 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ac2515f5-c3bc-4c83-be98-064fbdd018f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:24.113901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2515f5-c3bc-4c83-be98-064fbdd018f2 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:24.119191 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:24.119820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:24.120102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:24.120376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] PATH_INFO: `/v3/users/ab84fb2aa8d54a90b34d5ad239df96dc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:24.121545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab84fb2aa8d54a90b34d5ad239df96dc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:24.123460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:24.170512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab84fb2aa8d54a90b34d5ad239df96dc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.177838 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab84fb2aa8d54a90b34d5ad239df96dc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.178480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab84fb2aa8d54a90b34d5ad239df96dc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.179077 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab84fb2aa8d54a90b34d5ad239df96dc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.186269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab84fb2aa8d54a90b34d5ad239df96dc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.204138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1356/2695] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:24 2022] DELETE /identity/v3/users/ab84fb2aa8d54a90b34d5ad239df96dc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:24.208638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973270a-70e4-4a45-833b-43320a6eab0c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:24.209128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973270a-70e4-4a45-833b-43320a6eab0c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:24.209239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973270a-70e4-4a45-833b-43320a6eab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:24.228418 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9973270a-70e4-4a45-833b-43320a6eab0c None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:24.229656 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9973270a-70e4-4a45-833b-43320a6eab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:24.289330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1340/2696] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:24.299068 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:24.300448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:24.358688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c14aa-48cf-4a7a-b0a3-565603b4b37f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:24.362277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5cef8-f502-41f2-a6de-84c38a11199a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:24.362870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5cef8-f502-41f2-a6de-84c38a11199a service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:24.363143 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5cef8-f502-41f2-a6de-84c38a11199a service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:24.363411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5cef8-f502-41f2-a6de-84c38a11199a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:24.372900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5cef8-f502-41f2-a6de-84c38a11199a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:24.374317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5cef8-f502-41f2-a6de-84c38a11199a service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:24.384416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1357/2697] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:24.655576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973270a-70e4-4a45-833b-43320a6eab0c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:24.657121 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9973270a-70e4-4a45-833b-43320a6eab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:24.670538 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973270a-70e4-4a45-833b-43320a6eab0c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:24.678818 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:24.679495 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:24.679747 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:24.680020 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] PATH_INFO: `/v3/projects/61aff2beae19496ebe0e94ef70bcd350` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:24.681261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] RBAC: Authorizing `identity:delete_project(project_id=61aff2beae19496ebe0e94ef70bcd350)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:24.683217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:24.732204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61aff2beae19496ebe0e94ef70bcd350', 'request_id': 'req-613b8b59-8adf-480d-8307-7cd1d5ec336a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.737786 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61aff2beae19496ebe0e94ef70bcd350', 'request_id': 'req-613b8b59-8adf-480d-8307-7cd1d5ec336a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.744233 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61aff2beae19496ebe0e94ef70bcd350', 'request_id': 'req-613b8b59-8adf-480d-8307-7cd1d5ec336a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.756272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] The token cache is being invalidate because project 61aff2beae19496ebe0e94ef70bcd350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:24.756593 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.757207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.758667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1341/2698] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:24 2022] DELETE /identity/v3/projects/61aff2beae19496ebe0e94ef70bcd350 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:24.762906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5cef8-f502-41f2-a6de-84c38a11199a service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:24.773387 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5cef8-f502-41f2-a6de-84c38a11199a service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:24.794444 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:24.835653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430ecae-41bc-4380-a084-70dfb6eda14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:24.836191 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430ecae-41bc-4380-a084-70dfb6eda14b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:24.836469 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430ecae-41bc-4380-a084-70dfb6eda14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:24.836771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430ecae-41bc-4380-a084-70dfb6eda14b admin admin] PATH_INFO: `/v3/users/0bb589c3916a48b79a134228f8fcaee9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:24.837878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430ecae-41bc-4380-a084-70dfb6eda14b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb589c3916a48b79a134228f8fcaee9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:24.839828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430ecae-41bc-4380-a084-70dfb6eda14b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:24.887431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e430ecae-41bc-4380-a084-70dfb6eda14b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb589c3916a48b79a134228f8fcaee9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.894294 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e430ecae-41bc-4380-a084-70dfb6eda14b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb589c3916a48b79a134228f8fcaee9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.894961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e430ecae-41bc-4380-a084-70dfb6eda14b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb589c3916a48b79a134228f8fcaee9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.895573 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e430ecae-41bc-4380-a084-70dfb6eda14b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb589c3916a48b79a134228f8fcaee9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.903644 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e430ecae-41bc-4380-a084-70dfb6eda14b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb589c3916a48b79a134228f8fcaee9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:24.921341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1358/2699] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:24 2022] DELETE /identity/v3/users/0bb589c3916a48b79a134228f8fcaee9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:25.194954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:25.196436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:25.209957 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b8b59-8adf-480d-8307-7cd1d5ec336a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:25.269756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:25.270393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:25.270735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:25.271056 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] PATH_INFO: `/v3/projects/1bf2da7494554069b7edc1702c78bf3d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:25.272155 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf2da7494554069b7edc1702c78bf3d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:25.274102 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:25.285759 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:25.325163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf2da7494554069b7edc1702c78bf3d', 'request_id': 'req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:25.330569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf2da7494554069b7edc1702c78bf3d', 'request_id': 'req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:25.337002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf2da7494554069b7edc1702c78bf3d', 'request_id': 'req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:25.349453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] The token cache is being invalidate because project 1bf2da7494554069b7edc1702c78bf3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:25.349744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:25.350376 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-347141d0-70f5-45cc-b4e5-a25a27a4c8e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:25.351874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1342/2700] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:25 2022] DELETE /identity/v3/projects/1bf2da7494554069b7edc1702c78bf3d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:25.356813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62940b6b-37d0-4b31-be72-5c12ffffc37b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:25.357284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62940b6b-37d0-4b31-be72-5c12ffffc37b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:25.357537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62940b6b-37d0-4b31-be72-5c12ffffc37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:25.378068 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-62940b6b-37d0-4b31-be72-5c12ffffc37b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:25.379197 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-62940b6b-37d0-4b31-be72-5c12ffffc37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:25.437534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1359/2701] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:25.447631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa239e-4fde-4d95-827a-45f6651831d9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:25.447922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa239e-4fde-4d95-827a-45f6651831d9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:25.448200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa239e-4fde-4d95-827a-45f6651831d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:25.469638 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9ffa239e-4fde-4d95-827a-45f6651831d9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:25.470854 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9ffa239e-4fde-4d95-827a-45f6651831d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:25.479371 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1343/2702] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:25.483574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62940b6b-37d0-4b31-be72-5c12ffffc37b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:25.493836 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62940b6b-37d0-4b31-be72-5c12ffffc37b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:25.498555 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeced2-a2db-4654-842d-faa2924e662a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:25.499154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeced2-a2db-4654-842d-faa2924e662a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:25.499435 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeced2-a2db-4654-842d-faa2924e662a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:25.499708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeced2-a2db-4654-842d-faa2924e662a admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:25.500354 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeced2-a2db-4654-842d-faa2924e662a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:55:25.500696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeced2-a2db-4654-842d-faa2924e662a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:25.502349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeced2-a2db-4654-842d-faa2924e662a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:25.514981 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1360/2703] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:55:25.519267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa239e-4fde-4d95-827a-45f6651831d9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:25.529308 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa239e-4fde-4d95-827a-45f6651831d9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:25.534807 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727105cb-7fa4-4afd-a8e1-8745f32c6268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:25.535468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727105cb-7fa4-4afd-a8e1-8745f32c6268 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:25.535749 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727105cb-7fa4-4afd-a8e1-8745f32c6268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:25.536031 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727105cb-7fa4-4afd-a8e1-8745f32c6268 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:25.537284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727105cb-7fa4-4afd-a8e1-8745f32c6268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:25.539229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727105cb-7fa4-4afd-a8e1-8745f32c6268 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:25.573708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1344/2704] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:25 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:25.574923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeced2-a2db-4654-842d-faa2924e662a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:25.585303 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeced2-a2db-4654-842d-faa2924e662a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:25.611892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:25.645002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982a2e7-d74a-4f02-b8ee-e58712e2da0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:25.645552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982a2e7-d74a-4f02-b8ee-e58712e2da0b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:25.646094 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982a2e7-d74a-4f02-b8ee-e58712e2da0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:25.646370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982a2e7-d74a-4f02-b8ee-e58712e2da0b admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:25.646993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982a2e7-d74a-4f02-b8ee-e58712e2da0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:25.648941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982a2e7-d74a-4f02-b8ee-e58712e2da0b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:25.671006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1361/2705] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:25 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:25.675117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727105cb-7fa4-4afd-a8e1-8745f32c6268 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:25.686453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727105cb-7fa4-4afd-a8e1-8745f32c6268 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:25.691088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb642c2-db0b-4931-ba2b-e69c395344bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:25.691690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb642c2-db0b-4931-ba2b-e69c395344bf admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:25.692231 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb642c2-db0b-4931-ba2b-e69c395344bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:25.692536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb642c2-db0b-4931-ba2b-e69c395344bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:25.693445 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb642c2-db0b-4931-ba2b-e69c395344bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:25.695022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb642c2-db0b-4931-ba2b-e69c395344bf admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:25.704987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1345/2706] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:25.710288 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982a2e7-d74a-4f02-b8ee-e58712e2da0b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:25.720522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982a2e7-d74a-4f02-b8ee-e58712e2da0b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:25.726073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc5c38-4ea4-49f2-8399-eabb47a1985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:25.726670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc5c38-4ea4-49f2-8399-eabb47a1985a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:25.726947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc5c38-4ea4-49f2-8399-eabb47a1985a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:25.727214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc5c38-4ea4-49f2-8399-eabb47a1985a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:25.727847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc5c38-4ea4-49f2-8399-eabb47a1985a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:25.729459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc5c38-4ea4-49f2-8399-eabb47a1985a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:25.739207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1362/2707] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:25.743859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb642c2-db0b-4931-ba2b-e69c395344bf admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:25.756247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb642c2-db0b-4931-ba2b-e69c395344bf admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:25.760997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961152a-7bde-4033-9f5c-ec8b05ee2d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:25.761651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961152a-7bde-4033-9f5c-ec8b05ee2d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:25.761936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961152a-7bde-4033-9f5c-ec8b05ee2d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:25.762215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961152a-7bde-4033-9f5c-ec8b05ee2d57 admin admin] PATH_INFO: `/v3/projects/8edcb68eb092423ebcb94991eeeb3f17/users/850e23fa216247eeb7c0914c69f34f1f/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:25.776726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961152a-7bde-4033-9f5c-ec8b05ee2d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8edcb68eb092423ebcb94991eeeb3f17, user_id=850e23fa216247eeb7c0914c69f34f1f, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:25.779388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961152a-7bde-4033-9f5c-ec8b05ee2d57 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:25.792615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1346/2708] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:25 2022] PUT /identity/v3/projects/8edcb68eb092423ebcb94991eeeb3f17/users/850e23fa216247eeb7c0914c69f34f1f/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:25.803273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb2300-2e06-4b7d-bbba-14a68f03b1a0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:25.803559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb2300-2e06-4b7d-bbba-14a68f03b1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:25.803833 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb2300-2e06-4b7d-bbba-14a68f03b1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:25.831318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dddb2300-2e06-4b7d-bbba-14a68f03b1a0 None None] MFA Rules not processed for user `850e23fa216247eeb7c0914c69f34f1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:25.832560 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dddb2300-2e06-4b7d-bbba-14a68f03b1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:25.897013 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1363/2709] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:25.901305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961152a-7bde-4033-9f5c-ec8b05ee2d57 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:25.964701 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961152a-7bde-4033-9f5c-ec8b05ee2d57 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:25.969400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdf535-19b5-443a-b0b6-31e190dd36f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:25.970004 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdf535-19b5-443a-b0b6-31e190dd36f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:25.970289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdf535-19b5-443a-b0b6-31e190dd36f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:25.970568 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdf535-19b5-443a-b0b6-31e190dd36f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:25.971217 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdf535-19b5-443a-b0b6-31e190dd36f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:25.973165 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdf535-19b5-443a-b0b6-31e190dd36f3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:25.999654 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1347/2710] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:25 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:26.003731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb2300-2e06-4b7d-bbba-14a68f03b1a0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:26.015007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb2300-2e06-4b7d-bbba-14a68f03b1a0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:26.070516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093b78c-f9d2-42b5-9369-1ce4ae762767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:26.071114 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093b78c-f9d2-42b5-9369-1ce4ae762767 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.071391 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093b78c-f9d2-42b5-9369-1ce4ae762767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.071659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093b78c-f9d2-42b5-9369-1ce4ae762767 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.072277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093b78c-f9d2-42b5-9369-1ce4ae762767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:26.074223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093b78c-f9d2-42b5-9369-1ce4ae762767 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:26.094759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1364/2711] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:26 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:26.099570 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdf535-19b5-443a-b0b6-31e190dd36f3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:26.110335 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdf535-19b5-443a-b0b6-31e190dd36f3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:26.115003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21d272-291f-4343-85c9-0dc2d21ebf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:26.115591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21d272-291f-4343-85c9-0dc2d21ebf12 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.115870 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21d272-291f-4343-85c9-0dc2d21ebf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.116141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21d272-291f-4343-85c9-0dc2d21ebf12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.116816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21d272-291f-4343-85c9-0dc2d21ebf12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:26.118436 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21d272-291f-4343-85c9-0dc2d21ebf12 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:26.128826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1348/2712] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:26.132939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093b78c-f9d2-42b5-9369-1ce4ae762767 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:26.142910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093b78c-f9d2-42b5-9369-1ce4ae762767 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:26.148105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2490d8-04ef-410d-aa77-d50453d1b91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:26.148676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2490d8-04ef-410d-aa77-d50453d1b91f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.148963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2490d8-04ef-410d-aa77-d50453d1b91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.149232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2490d8-04ef-410d-aa77-d50453d1b91f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.149866 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2490d8-04ef-410d-aa77-d50453d1b91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:26.151483 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2490d8-04ef-410d-aa77-d50453d1b91f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:26.161418 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1365/2713] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:26.166003 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21d272-291f-4343-85c9-0dc2d21ebf12 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:26.175899 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21d272-291f-4343-85c9-0dc2d21ebf12 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:26.180675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624677f1-6295-4c4d-915e-85c7ae18dd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:26.181332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624677f1-6295-4c4d-915e-85c7ae18dd69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.181618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624677f1-6295-4c4d-915e-85c7ae18dd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.181896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624677f1-6295-4c4d-915e-85c7ae18dd69 admin admin] PATH_INFO: `/v3/projects/85c6bd3f11134100b97a3c62571969c4/users/588d2dd8731341a1bbb44c02750de3fe/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.189811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:26.197897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624677f1-6295-4c4d-915e-85c7ae18dd69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c6bd3f11134100b97a3c62571969c4, user_id=588d2dd8731341a1bbb44c02750de3fe, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:26.201079 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624677f1-6295-4c4d-915e-85c7ae18dd69 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:26.215047 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1349/2714] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:26 2022] PUT /identity/v3/projects/85c6bd3f11134100b97a3c62571969c4/users/588d2dd8731341a1bbb44c02750de3fe/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:26.225713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce54ae-b218-4065-a663-f897e5fdaf08 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.226003 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce54ae-b218-4065-a663-f897e5fdaf08 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.226359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce54ae-b218-4065-a663-f897e5fdaf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.254083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-99ce54ae-b218-4065-a663-f897e5fdaf08 None None] MFA Rules not processed for user `588d2dd8731341a1bbb44c02750de3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:26.255427 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-99ce54ae-b218-4065-a663-f897e5fdaf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:26.259570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:26.324230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1366/2715] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:26.328792 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624677f1-6295-4c4d-915e-85c7ae18dd69 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:26.391390 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624677f1-6295-4c4d-915e-85c7ae18dd69 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:26.397140 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cacfd7-4dc4-4c3a-9060-f5d0f399287c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:26.397750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cacfd7-4dc4-4c3a-9060-f5d0f399287c admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.398040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cacfd7-4dc4-4c3a-9060-f5d0f399287c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.398374 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cacfd7-4dc4-4c3a-9060-f5d0f399287c admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.398947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cacfd7-4dc4-4c3a-9060-f5d0f399287c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:26.400911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cacfd7-4dc4-4c3a-9060-f5d0f399287c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:26.427601 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1350/2716] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:26 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:26.431683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce54ae-b218-4065-a663-f897e5fdaf08 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:26.443910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce54ae-b218-4065-a663-f897e5fdaf08 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:26.500625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d63e78-1b55-470f-8bd5-5d3f7bb99f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:26.501224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d63e78-1b55-470f-8bd5-5d3f7bb99f62 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.501506 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d63e78-1b55-470f-8bd5-5d3f7bb99f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.501777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d63e78-1b55-470f-8bd5-5d3f7bb99f62 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.502412 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d63e78-1b55-470f-8bd5-5d3f7bb99f62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:26.504379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d63e78-1b55-470f-8bd5-5d3f7bb99f62 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:26.526216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1367/2717] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:26 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:26.530243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cacfd7-4dc4-4c3a-9060-f5d0f399287c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:26.542848 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cacfd7-4dc4-4c3a-9060-f5d0f399287c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:26.547820 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc796071-0a7b-4840-8204-5032d2c456b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:26.548428 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc796071-0a7b-4840-8204-5032d2c456b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.548746 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc796071-0a7b-4840-8204-5032d2c456b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.549027 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc796071-0a7b-4840-8204-5032d2c456b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.549675 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc796071-0a7b-4840-8204-5032d2c456b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:26.551276 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc796071-0a7b-4840-8204-5032d2c456b5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:26.562543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1351/2718] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:26.567209 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d63e78-1b55-470f-8bd5-5d3f7bb99f62 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:26.577734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d63e78-1b55-470f-8bd5-5d3f7bb99f62 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:26.582368 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9501a8-6c26-4233-99c9-8d35c1fa7eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:26.583008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9501a8-6c26-4233-99c9-8d35c1fa7eed admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.583378 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9501a8-6c26-4233-99c9-8d35c1fa7eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.583657 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9501a8-6c26-4233-99c9-8d35c1fa7eed admin admin] PATH_INFO: `/v3/projects/c47feb7b72264bb5a89161a2e52c0313/users/09f37f3ddcea45ec96bc7ced75cf21e3/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.598691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9501a8-6c26-4233-99c9-8d35c1fa7eed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47feb7b72264bb5a89161a2e52c0313, user_id=09f37f3ddcea45ec96bc7ced75cf21e3, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:26.601419 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9501a8-6c26-4233-99c9-8d35c1fa7eed admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:26.615637 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1368/2719] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:26 2022] PUT /identity/v3/projects/c47feb7b72264bb5a89161a2e52c0313/users/09f37f3ddcea45ec96bc7ced75cf21e3/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:26.619651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc796071-0a7b-4840-8204-5032d2c456b5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:26.629511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc796071-0a7b-4840-8204-5032d2c456b5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:26.691645 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20b41b-8070-4100-826a-1cd549831510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:26.692558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20b41b-8070-4100-826a-1cd549831510 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.693450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20b41b-8070-4100-826a-1cd549831510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.693741 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20b41b-8070-4100-826a-1cd549831510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.694385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20b41b-8070-4100-826a-1cd549831510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:26.695988 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20b41b-8070-4100-826a-1cd549831510 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:26.706214 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1352/2720] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:26.710371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9501a8-6c26-4233-99c9-8d35c1fa7eed admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:26.720480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9501a8-6c26-4233-99c9-8d35c1fa7eed admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:26.725180 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db99a35-c6ec-4a91-914d-dbb40bd9b4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:26.725823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db99a35-c6ec-4a91-914d-dbb40bd9b4c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.726106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db99a35-c6ec-4a91-914d-dbb40bd9b4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.726382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db99a35-c6ec-4a91-914d-dbb40bd9b4c9 admin admin] PATH_INFO: `/v3/projects/c47feb7b72264bb5a89161a2e52c0313/users/09f37f3ddcea45ec96bc7ced75cf21e3/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.728313 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db99a35-c6ec-4a91-914d-dbb40bd9b4c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47feb7b72264bb5a89161a2e52c0313, user_id=09f37f3ddcea45ec96bc7ced75cf21e3, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:26.730992 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db99a35-c6ec-4a91-914d-dbb40bd9b4c9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:26.741653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1369/2721] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:26 2022] PUT /identity/v3/projects/c47feb7b72264bb5a89161a2e52c0313/users/09f37f3ddcea45ec96bc7ced75cf21e3/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:26.752616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f5aa-4212-411c-8bca-be4df4528e4b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:26.752906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f5aa-4212-411c-8bca-be4df4528e4b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:26.753184 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f5aa-4212-411c-8bca-be4df4528e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:26.907021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-44f9f5aa-4212-411c-8bca-be4df4528e4b None None] MFA Rules not processed for user `09f37f3ddcea45ec96bc7ced75cf21e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:26.908283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-44f9f5aa-4212-411c-8bca-be4df4528e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:26.956990 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:26.983144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1353/2722] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:26.994328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db99a35-c6ec-4a91-914d-dbb40bd9b4c9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:26.996044 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0db99a35-c6ec-4a91-914d-dbb40bd9b4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:27.057041 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db99a35-c6ec-4a91-914d-dbb40bd9b4c9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:27.060664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc22bd-bd7f-4510-9bb8-250b6d2eb03d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:27.061262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc22bd-bd7f-4510-9bb8-250b6d2eb03d service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:27.061542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc22bd-bd7f-4510-9bb8-250b6d2eb03d service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:27.061813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc22bd-bd7f-4510-9bb8-250b6d2eb03d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:27.071479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc22bd-bd7f-4510-9bb8-250b6d2eb03d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:27.072940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc22bd-bd7f-4510-9bb8-250b6d2eb03d service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:27.077809 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:27.082989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1370/2723] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:27.125789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9f5aa-4212-411c-8bca-be4df4528e4b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:27.127209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-44f9f5aa-4212-411c-8bca-be4df4528e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:27.139927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9f5aa-4212-411c-8bca-be4df4528e4b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:27.143479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cef6fc-fb3f-49b9-b039-a1df8967df46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:27.144077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef6fc-fb3f-49b9-b039-a1df8967df46 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:27.144357 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef6fc-fb3f-49b9-b039-a1df8967df46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:27.144664 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef6fc-fb3f-49b9-b039-a1df8967df46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:27.203720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cef6fc-fb3f-49b9-b039-a1df8967df46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:27.205124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cef6fc-fb3f-49b9-b039-a1df8967df46 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:27.214608 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1354/2724] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:30.116748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc22bd-bd7f-4510-9bb8-250b6d2eb03d service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:30.181546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc22bd-bd7f-4510-9bb8-250b6d2eb03d service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:30.186026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:30.186652 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:30.186932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:30.187210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] PATH_INFO: `/v3/users/850e23fa216247eeb7c0914c69f34f1f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:30.188321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=850e23fa216247eeb7c0914c69f34f1f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:30.190353 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:30.239530 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '850e23fa216247eeb7c0914c69f34f1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:30.246941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '850e23fa216247eeb7c0914c69f34f1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:30.247648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '850e23fa216247eeb7c0914c69f34f1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:30.248256 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '850e23fa216247eeb7c0914c69f34f1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:30.256664 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '850e23fa216247eeb7c0914c69f34f1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:30.275810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1371/2725] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:30 2022] DELETE /identity/v3/users/850e23fa216247eeb7c0914c69f34f1f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:30.280952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaf2b5-340c-4de0-97d0-ab4157a38e03 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:30.281478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaf2b5-340c-4de0-97d0-ab4157a38e03 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:30.281751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaf2b5-340c-4de0-97d0-ab4157a38e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:30.305286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-46eaf2b5-340c-4de0-97d0-ab4157a38e03 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:30.306522 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-46eaf2b5-340c-4de0-97d0-ab4157a38e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:30.366309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1355/2726] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:30.376061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:30.377624 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:30.454480 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe019e2-2615-41ae-99bd-0e43eb419556 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:30.459409 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806050dd-e224-4ae1-8072-45719504d7b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:30.460265 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806050dd-e224-4ae1-8072-45719504d7b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:30.460686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806050dd-e224-4ae1-8072-45719504d7b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:30.461058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806050dd-e224-4ae1-8072-45719504d7b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:30.471397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806050dd-e224-4ae1-8072-45719504d7b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:30.472902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806050dd-e224-4ae1-8072-45719504d7b8 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:30.483135 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1372/2727] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:30.776164 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eaf2b5-340c-4de0-97d0-ab4157a38e03 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:30.778367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-46eaf2b5-340c-4de0-97d0-ab4157a38e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:30.791688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eaf2b5-340c-4de0-97d0-ab4157a38e03 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:30.796845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:30.797564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:30.797922 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:30.798264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] PATH_INFO: `/v3/projects/8edcb68eb092423ebcb94991eeeb3f17` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:30.799479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=8edcb68eb092423ebcb94991eeeb3f17)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:30.801529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:30.856788 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8edcb68eb092423ebcb94991eeeb3f17', 'request_id': 'req-e676cf56-81ac-424e-82b2-2a96e6e27cee'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:30.864028 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8edcb68eb092423ebcb94991eeeb3f17', 'request_id': 'req-e676cf56-81ac-424e-82b2-2a96e6e27cee'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:30.871356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8edcb68eb092423ebcb94991eeeb3f17', 'request_id': 'req-e676cf56-81ac-424e-82b2-2a96e6e27cee'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:30.884604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] The token cache is being invalidate because project 8edcb68eb092423ebcb94991eeeb3f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:30.884899 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:30.885536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:30.887016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1356/2728] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:30 2022] DELETE /identity/v3/projects/8edcb68eb092423ebcb94991eeeb3f17 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:30.891058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806050dd-e224-4ae1-8072-45719504d7b8 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:30.901551 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806050dd-e224-4ae1-8072-45719504d7b8 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:30.920148 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:30.963665 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:30.964327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:30.964648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:30.964945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] PATH_INFO: `/v3/users/588d2dd8731341a1bbb44c02750de3fe` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:30.966106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=588d2dd8731341a1bbb44c02750de3fe)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:30.968029 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:31.025545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '588d2dd8731341a1bbb44c02750de3fe'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.032626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '588d2dd8731341a1bbb44c02750de3fe'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.033306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '588d2dd8731341a1bbb44c02750de3fe'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.033915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '588d2dd8731341a1bbb44c02750de3fe'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.041574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '588d2dd8731341a1bbb44c02750de3fe'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.059280 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1373/2729] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:30 2022] DELETE /identity/v3/users/588d2dd8731341a1bbb44c02750de3fe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:55:31.340948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:31.342405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:31.352537 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:31.356529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676cf56-81ac-424e-82b2-2a96e6e27cee admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:31.421968 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:31.422609 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:31.422894 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:31.423171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] PATH_INFO: `/v3/projects/85c6bd3f11134100b97a3c62571969c4` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:31.424312 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c6bd3f11134100b97a3c62571969c4)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:31.426273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:31.477913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c6bd3f11134100b97a3c62571969c4', 'request_id': 'req-66afadd9-6a09-4b7e-89b3-711f534badd5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.483577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c6bd3f11134100b97a3c62571969c4', 'request_id': 'req-66afadd9-6a09-4b7e-89b3-711f534badd5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.491432 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c6bd3f11134100b97a3c62571969c4', 'request_id': 'req-66afadd9-6a09-4b7e-89b3-711f534badd5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.504910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] The token cache is being invalidate because project 85c6bd3f11134100b97a3c62571969c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:31.505882 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.506561 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.508033 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1357/2730] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:31 2022] DELETE /identity/v3/projects/85c6bd3f11134100b97a3c62571969c4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:31.512130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:31.513598 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:31.526451 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537499f2-2978-4baa-9551-3f8c86401bae admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:31.585030 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58c399-ec28-47f1-a09c-d046cb9838c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:31.585653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58c399-ec28-47f1-a09c-d046cb9838c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:31.585933 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58c399-ec28-47f1-a09c-d046cb9838c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:31.586203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58c399-ec28-47f1-a09c-d046cb9838c6 admin admin] PATH_INFO: `/v3/users/09f37f3ddcea45ec96bc7ced75cf21e3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:31.587318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58c399-ec28-47f1-a09c-d046cb9838c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f37f3ddcea45ec96bc7ced75cf21e3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:31.589287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58c399-ec28-47f1-a09c-d046cb9838c6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:31.647636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:31.652100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8e58c399-ec28-47f1-a09c-d046cb9838c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f37f3ddcea45ec96bc7ced75cf21e3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.663878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8e58c399-ec28-47f1-a09c-d046cb9838c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f37f3ddcea45ec96bc7ced75cf21e3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.664634 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8e58c399-ec28-47f1-a09c-d046cb9838c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f37f3ddcea45ec96bc7ced75cf21e3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.665312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8e58c399-ec28-47f1-a09c-d046cb9838c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f37f3ddcea45ec96bc7ced75cf21e3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.680092 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8e58c399-ec28-47f1-a09c-d046cb9838c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f37f3ddcea45ec96bc7ced75cf21e3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:31.696287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1374/2731] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:31 2022] DELETE /identity/v3/users/09f37f3ddcea45ec96bc7ced75cf21e3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:32.003842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:32.005979 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:32.019626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afadd9-6a09-4b7e-89b3-711f534badd5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:32.085086 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:32.085727 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:32.086013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:32.086296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] PATH_INFO: `/v3/projects/c47feb7b72264bb5a89161a2e52c0313` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:32.087439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c47feb7b72264bb5a89161a2e52c0313)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:32.089409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:32.130594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:32.141076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c47feb7b72264bb5a89161a2e52c0313', 'request_id': 'req-47555d03-0a22-47d9-a901-ea05f84d8db2'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:32.147149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c47feb7b72264bb5a89161a2e52c0313', 'request_id': 'req-47555d03-0a22-47d9-a901-ea05f84d8db2'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:32.153280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c47feb7b72264bb5a89161a2e52c0313', 'request_id': 'req-47555d03-0a22-47d9-a901-ea05f84d8db2'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:32.166505 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] The token cache is being invalidate because project c47feb7b72264bb5a89161a2e52c0313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:32.166792 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:32.167405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-47555d03-0a22-47d9-a901-ea05f84d8db2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:32.168898 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1358/2732] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:32 2022] DELETE /identity/v3/projects/c47feb7b72264bb5a89161a2e52c0313 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:32.175400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c51887-31e5-4ccc-9167-eeffa1b0db26 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:32.175686 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c51887-31e5-4ccc-9167-eeffa1b0db26 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:32.175961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c51887-31e5-4ccc-9167-eeffa1b0db26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:32.195849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-12c51887-31e5-4ccc-9167-eeffa1b0db26 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:32.197084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-12c51887-31e5-4ccc-9167-eeffa1b0db26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:32.273364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1375/2733] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:32.283685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6210823-82e3-457b-a7b3-2bb212b450c5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:32.284093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6210823-82e3-457b-a7b3-2bb212b450c5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:32.284383 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6210823-82e3-457b-a7b3-2bb212b450c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:32.309655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b6210823-82e3-457b-a7b3-2bb212b450c5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:32.311141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b6210823-82e3-457b-a7b3-2bb212b450c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:32.321811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1359/2734] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:32.326630 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c51887-31e5-4ccc-9167-eeffa1b0db26 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:32.337102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c51887-31e5-4ccc-9167-eeffa1b0db26 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:32.341732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c07fd-29ab-40cf-b65a-010103f96615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:32.342332 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c07fd-29ab-40cf-b65a-010103f96615 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:32.342616 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c07fd-29ab-40cf-b65a-010103f96615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:32.342920 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c07fd-29ab-40cf-b65a-010103f96615 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:32.343564 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c07fd-29ab-40cf-b65a-010103f96615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:55:32.343871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c07fd-29ab-40cf-b65a-010103f96615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:32.345479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c07fd-29ab-40cf-b65a-010103f96615 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:32.357327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1376/2735] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:55:32.362535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6210823-82e3-457b-a7b3-2bb212b450c5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:32.372631 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6210823-82e3-457b-a7b3-2bb212b450c5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:32.377367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e66d6-9cd0-422c-b834-d0b0a3548157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:32.377965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e66d6-9cd0-422c-b834-d0b0a3548157 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:32.378247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e66d6-9cd0-422c-b834-d0b0a3548157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:32.378526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e66d6-9cd0-422c-b834-d0b0a3548157 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:32.379150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e66d6-9cd0-422c-b834-d0b0a3548157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:32.382062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e66d6-9cd0-422c-b834-d0b0a3548157 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:32.411117 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1360/2736] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:32 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:32.415799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c07fd-29ab-40cf-b65a-010103f96615 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:32.426641 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c07fd-29ab-40cf-b65a-010103f96615 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:32.486907 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213641e-d0d0-4757-b8f6-c7333162aa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:32.487572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213641e-d0d0-4757-b8f6-c7333162aa62 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:32.487793 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213641e-d0d0-4757-b8f6-c7333162aa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:32.488089 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213641e-d0d0-4757-b8f6-c7333162aa62 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:32.488739 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213641e-d0d0-4757-b8f6-c7333162aa62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:32.490993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213641e-d0d0-4757-b8f6-c7333162aa62 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:32.514377 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1377/2737] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:32 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:55:32.519417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e66d6-9cd0-422c-b834-d0b0a3548157 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:32.530672 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e66d6-9cd0-422c-b834-d0b0a3548157 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:32.535378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1631d-9280-46d3-bbc2-f011e397103c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:32.535911 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1631d-9280-46d3-bbc2-f011e397103c admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:32.536253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1631d-9280-46d3-bbc2-f011e397103c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:32.536815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1631d-9280-46d3-bbc2-f011e397103c admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:32.537511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1631d-9280-46d3-bbc2-f011e397103c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:32.539109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1631d-9280-46d3-bbc2-f011e397103c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:32.549999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1361/2738] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:32.554896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213641e-d0d0-4757-b8f6-c7333162aa62 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:32.568058 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213641e-d0d0-4757-b8f6-c7333162aa62 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:32.572853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dee87-d844-41ec-8fc1-9b405c9b7c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:32.573474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dee87-d844-41ec-8fc1-9b405c9b7c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:32.573748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dee87-d844-41ec-8fc1-9b405c9b7c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:32.574015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dee87-d844-41ec-8fc1-9b405c9b7c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:32.574652 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dee87-d844-41ec-8fc1-9b405c9b7c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:32.576224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dee87-d844-41ec-8fc1-9b405c9b7c7f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:32.587620 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1378/2739] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:32.592926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1631d-9280-46d3-bbc2-f011e397103c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:32.607612 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1631d-9280-46d3-bbc2-f011e397103c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:32.615344 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a7210-05b9-4053-9f75-77c36f3f3396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:32.616379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a7210-05b9-4053-9f75-77c36f3f3396 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:32.616985 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a7210-05b9-4053-9f75-77c36f3f3396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:32.617506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a7210-05b9-4053-9f75-77c36f3f3396 admin admin] PATH_INFO: `/v3/projects/4b26a99e83114afcb86e6b31afbea0ae/users/72900ecaf4514aa39b9d98e45d1ecaa1/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:32.646640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a7210-05b9-4053-9f75-77c36f3f3396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b26a99e83114afcb86e6b31afbea0ae, user_id=72900ecaf4514aa39b9d98e45d1ecaa1, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:32.651477 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a7210-05b9-4053-9f75-77c36f3f3396 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:32.668780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1362/2740] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:32 2022] PUT /identity/v3/projects/4b26a99e83114afcb86e6b31afbea0ae/users/72900ecaf4514aa39b9d98e45d1ecaa1/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:32.684897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94eef94-4ef4-4eda-ae1d-3ea98b2e4d4e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:32.685215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94eef94-4ef4-4eda-ae1d-3ea98b2e4d4e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:32.685494 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94eef94-4ef4-4eda-ae1d-3ea98b2e4d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:32.726024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d94eef94-4ef4-4eda-ae1d-3ea98b2e4d4e None None] MFA Rules not processed for user `72900ecaf4514aa39b9d98e45d1ecaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:32.728333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d94eef94-4ef4-4eda-ae1d-3ea98b2e4d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:32.747192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:32.835854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1379/2741] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:32.841906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a7210-05b9-4053-9f75-77c36f3f3396 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:32.922896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a7210-05b9-4053-9f75-77c36f3f3396 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:32.928603 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e765142-5165-4f4b-8645-f98dcd2197b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:32.929238 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e765142-5165-4f4b-8645-f98dcd2197b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:32.929560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e765142-5165-4f4b-8645-f98dcd2197b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:32.929919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e765142-5165-4f4b-8645-f98dcd2197b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:32.930602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e765142-5165-4f4b-8645-f98dcd2197b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:32.932722 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e765142-5165-4f4b-8645-f98dcd2197b1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:32.967482 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1363/2742] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:32 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:32.971837 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94eef94-4ef4-4eda-ae1d-3ea98b2e4d4e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:32.983603 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94eef94-4ef4-4eda-ae1d-3ea98b2e4d4e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:33.064382 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbd278-7fc3-4828-9b22-f4dd1f29cca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:33.065264 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbd278-7fc3-4828-9b22-f4dd1f29cca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:33.065687 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbd278-7fc3-4828-9b22-f4dd1f29cca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:33.066017 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbd278-7fc3-4828-9b22-f4dd1f29cca5 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:33.067061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbd278-7fc3-4828-9b22-f4dd1f29cca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:33.070016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbd278-7fc3-4828-9b22-f4dd1f29cca5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:33.101931 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1380/2743] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:32 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:33.106885 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e765142-5165-4f4b-8645-f98dcd2197b1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:33.121529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e765142-5165-4f4b-8645-f98dcd2197b1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:33.127476 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a95fd-dde4-410d-b385-50bd3b5419c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:33.128239 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a95fd-dde4-410d-b385-50bd3b5419c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:33.128602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a95fd-dde4-410d-b385-50bd3b5419c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:33.128942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a95fd-dde4-410d-b385-50bd3b5419c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:33.130614 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a95fd-dde4-410d-b385-50bd3b5419c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:33.133160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a95fd-dde4-410d-b385-50bd3b5419c4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:33.152349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1364/2744] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:33.156736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbd278-7fc3-4828-9b22-f4dd1f29cca5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:33.167913 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbd278-7fc3-4828-9b22-f4dd1f29cca5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:33.173335 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896a27f-e8a8-464b-8db6-e15051c2c16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:33.174000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896a27f-e8a8-464b-8db6-e15051c2c16d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:33.177966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896a27f-e8a8-464b-8db6-e15051c2c16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:33.178215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896a27f-e8a8-464b-8db6-e15051c2c16d admin admin] PATH_INFO: `/v3/projects/73c1eb31d0ac48a4b0adb5bc725cd186/users/887d6dfb244846809e35ffd91f7ca20f/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:33.195726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896a27f-e8a8-464b-8db6-e15051c2c16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c1eb31d0ac48a4b0adb5bc725cd186, user_id=887d6dfb244846809e35ffd91f7ca20f, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:33.198487 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896a27f-e8a8-464b-8db6-e15051c2c16d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:33.218844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1381/2745] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:33 2022] PUT /identity/v3/projects/73c1eb31d0ac48a4b0adb5bc725cd186/users/887d6dfb244846809e35ffd91f7ca20f/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:33.224385 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a95fd-dde4-410d-b385-50bd3b5419c4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:33.237322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a95fd-dde4-410d-b385-50bd3b5419c4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:33.277763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:33.313666 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2cd4f-3d05-4737-8326-f6365338f4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:33.314814 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2cd4f-3d05-4737-8326-f6365338f4fc admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:33.316222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2cd4f-3d05-4737-8326-f6365338f4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:33.316222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2cd4f-3d05-4737-8326-f6365338f4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:33.316222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2cd4f-3d05-4737-8326-f6365338f4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:33.318547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2cd4f-3d05-4737-8326-f6365338f4fc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:33.337625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1365/2746] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:33.343831 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896a27f-e8a8-464b-8db6-e15051c2c16d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:33.357283 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896a27f-e8a8-464b-8db6-e15051c2c16d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:33.362536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc250722-5cc6-477e-82f0-54487ebd13cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:33.363206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc250722-5cc6-477e-82f0-54487ebd13cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:33.363481 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc250722-5cc6-477e-82f0-54487ebd13cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:33.363751 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc250722-5cc6-477e-82f0-54487ebd13cc admin admin] PATH_INFO: `/v3/projects/73c1eb31d0ac48a4b0adb5bc725cd186/users/887d6dfb244846809e35ffd91f7ca20f/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:33.365895 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc250722-5cc6-477e-82f0-54487ebd13cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c1eb31d0ac48a4b0adb5bc725cd186, user_id=887d6dfb244846809e35ffd91f7ca20f, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:33.369020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc250722-5cc6-477e-82f0-54487ebd13cc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:33.383578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1382/2747] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:33 2022] PUT /identity/v3/projects/73c1eb31d0ac48a4b0adb5bc725cd186/users/887d6dfb244846809e35ffd91f7ca20f/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:33.396469 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0326bd-f752-4981-83ac-f5433e4b5ce0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:33.397124 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0326bd-f752-4981-83ac-f5433e4b5ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:33.397213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0326bd-f752-4981-83ac-f5433e4b5ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:33.430929 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ff0326bd-f752-4981-83ac-f5433e4b5ce0 None None] MFA Rules not processed for user `887d6dfb244846809e35ffd91f7ca20f`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:33.432146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ff0326bd-f752-4981-83ac-f5433e4b5ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:33.514841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1366/2748] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:33.530304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc250722-5cc6-477e-82f0-54487ebd13cc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:33.532143 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dc250722-5cc6-477e-82f0-54487ebd13cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:33.611984 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc250722-5cc6-477e-82f0-54487ebd13cc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:33.618012 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83efb90-a92f-4dcb-9ebd-fe9e73a6d037 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:33.618963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83efb90-a92f-4dcb-9ebd-fe9e73a6d037 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:33.619445 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83efb90-a92f-4dcb-9ebd-fe9e73a6d037 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:33.619923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83efb90-a92f-4dcb-9ebd-fe9e73a6d037 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:33.633077 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83efb90-a92f-4dcb-9ebd-fe9e73a6d037 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:33.635037 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83efb90-a92f-4dcb-9ebd-fe9e73a6d037 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:33.650610 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1383/2749] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:34.318793 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0326bd-f752-4981-83ac-f5433e4b5ce0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:34.381700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0326bd-f752-4981-83ac-f5433e4b5ce0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:34.386930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:34.387562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:34.387845 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:34.388119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] PATH_INFO: `/v3/users/72900ecaf4514aa39b9d98e45d1ecaa1` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:34.389296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] RBAC: Authorizing `identity:delete_user(user_id=72900ecaf4514aa39b9d98e45d1ecaa1)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:34.391288 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:34.446235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72900ecaf4514aa39b9d98e45d1ecaa1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:34.446958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72900ecaf4514aa39b9d98e45d1ecaa1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:34.453960 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72900ecaf4514aa39b9d98e45d1ecaa1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:34.456455 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:34.462168 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72900ecaf4514aa39b9d98e45d1ecaa1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:34.462923 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72900ecaf4514aa39b9d98e45d1ecaa1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:34.481855 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1367/2750] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:34 2022] DELETE /identity/v3/users/72900ecaf4514aa39b9d98e45d1ecaa1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:34.486655 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb0a16-f9e2-43af-9690-14a444efe5e5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:34.487026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb0a16-f9e2-43af-9690-14a444efe5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:34.487347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb0a16-f9e2-43af-9690-14a444efe5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:34.509072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a7cb0a16-f9e2-43af-9690-14a444efe5e5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:34.510364 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a7cb0a16-f9e2-43af-9690-14a444efe5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:34.515497 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:34.579346 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1384/2751] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:34.589687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:34.591109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:34.653248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb54bb-0776-41e4-8266-30c80562116c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:34.657090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd53c05-bf1f-4f01-864e-c113af5394c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:34.657699 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd53c05-bf1f-4f01-864e-c113af5394c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:34.657980 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd53c05-bf1f-4f01-864e-c113af5394c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:34.658260 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd53c05-bf1f-4f01-864e-c113af5394c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:34.671930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd53c05-bf1f-4f01-864e-c113af5394c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:34.673533 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd53c05-bf1f-4f01-864e-c113af5394c3 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:34.690443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1368/2752] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:34.915769 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb0a16-f9e2-43af-9690-14a444efe5e5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:34.917784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a7cb0a16-f9e2-43af-9690-14a444efe5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:34.936379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb0a16-f9e2-43af-9690-14a444efe5e5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:34.944177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc46ae-1476-48e2-a32d-1b1c630e3f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:34.945002 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc46ae-1476-48e2-a32d-1b1c630e3f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:34.945410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc46ae-1476-48e2-a32d-1b1c630e3f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:34.945800 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc46ae-1476-48e2-a32d-1b1c630e3f5c admin admin] PATH_INFO: `/v3/users/b1e3571c1f31481b9985337fde1fc953` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:34.947403 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc46ae-1476-48e2-a32d-1b1c630e3f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1e3571c1f31481b9985337fde1fc953)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:34.949645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc46ae-1476-48e2-a32d-1b1c630e3f5c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:34.983438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd53c05-bf1f-4f01-864e-c113af5394c3 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:34.985448 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bbd53c05-bf1f-4f01-864e-c113af5394c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:35.000395 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd53c05-bf1f-4f01-864e-c113af5394c3 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:35.007014 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:35.007637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:35.007965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:35.008340 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] PATH_INFO: `/v3/projects/4b26a99e83114afcb86e6b31afbea0ae` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:35.009606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b26a99e83114afcb86e6b31afbea0ae)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:35.011706 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:35.018688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f8bc46ae-1476-48e2-a32d-1b1c630e3f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1e3571c1f31481b9985337fde1fc953'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.031744 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f8bc46ae-1476-48e2-a32d-1b1c630e3f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1e3571c1f31481b9985337fde1fc953'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.032533 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f8bc46ae-1476-48e2-a32d-1b1c630e3f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1e3571c1f31481b9985337fde1fc953'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.033154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f8bc46ae-1476-48e2-a32d-1b1c630e3f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1e3571c1f31481b9985337fde1fc953'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.042927 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f8bc46ae-1476-48e2-a32d-1b1c630e3f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1e3571c1f31481b9985337fde1fc953'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.062148 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1385/2753] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:34 2022] DELETE /identity/v3/users/b1e3571c1f31481b9985337fde1fc953 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:35.066916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa091407-46fb-4dc0-b113-fc510b439227 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:35.067961 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa091407-46fb-4dc0-b113-fc510b439227 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:35.068247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa091407-46fb-4dc0-b113-fc510b439227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:35.068457 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b26a99e83114afcb86e6b31afbea0ae', 'request_id': 'req-c31210b4-47d8-4272-998b-c34e50c6de89'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.074296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b26a99e83114afcb86e6b31afbea0ae', 'request_id': 'req-c31210b4-47d8-4272-998b-c34e50c6de89'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.081811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b26a99e83114afcb86e6b31afbea0ae', 'request_id': 'req-c31210b4-47d8-4272-998b-c34e50c6de89'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.088865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fa091407-46fb-4dc0-b113-fc510b439227 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:35.090082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fa091407-46fb-4dc0-b113-fc510b439227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:35.095954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] The token cache is being invalidate because project 4b26a99e83114afcb86e6b31afbea0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:35.096321 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.097058 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.099175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1369/2754] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:34 2022] DELETE /identity/v3/projects/4b26a99e83114afcb86e6b31afbea0ae => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:35.104329 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:35.106563 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:35.157993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1386/2755] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:35.168749 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa091407-46fb-4dc0-b113-fc510b439227 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:35.170115 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fa091407-46fb-4dc0-b113-fc510b439227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:35.175864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31210b4-47d8-4272-998b-c34e50c6de89 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:35.181599 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:35.182236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:35.182514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:35.182790 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] PATH_INFO: `/v3/users/887d6dfb244846809e35ffd91f7ca20f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:35.184088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=887d6dfb244846809e35ffd91f7ca20f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:35.186222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:35.189149 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:35.205559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:35.248088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa091407-46fb-4dc0-b113-fc510b439227 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:35.248953 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '887d6dfb244846809e35ffd91f7ca20f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.249647 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '887d6dfb244846809e35ffd91f7ca20f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.251908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643dd64e-2619-4730-a246-44d021bf486b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:35.252541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dd64e-2619-4730-a246-44d021bf486b service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:35.252827 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dd64e-2619-4730-a246-44d021bf486b service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:35.253100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dd64e-2619-4730-a246-44d021bf486b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:35.256838 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '887d6dfb244846809e35ffd91f7ca20f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.263243 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643dd64e-2619-4730-a246-44d021bf486b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:35.264713 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643dd64e-2619-4730-a246-44d021bf486b service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:35.266061 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '887d6dfb244846809e35ffd91f7ca20f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.266919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '887d6dfb244846809e35ffd91f7ca20f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.276155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1387/2756] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:55:35.295152 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1370/2757] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:35 2022] DELETE /identity/v3/users/887d6dfb244846809e35ffd91f7ca20f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:35.650871 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643dd64e-2619-4730-a246-44d021bf486b service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:35.652432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-643dd64e-2619-4730-a246-44d021bf486b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:35.667254 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643dd64e-2619-4730-a246-44d021bf486b service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:35.686189 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:35.688077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:35.744402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:35.745165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:35.745452 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:35.745730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] PATH_INFO: `/v3/projects/0ee5a24e009543a38e66a262fae177b9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:35.746918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee5a24e009543a38e66a262fae177b9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:35.748868 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:35.776356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df52ff-7e37-4b49-b245-756dbd3ddc76 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:35.782098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:35.786879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:35.786879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:35.786879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] PATH_INFO: `/v3/projects/73c1eb31d0ac48a4b0adb5bc725cd186` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:35.786879 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73c1eb31d0ac48a4b0adb5bc725cd186)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:35.790569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:35.810217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee5a24e009543a38e66a262fae177b9', 'request_id': 'req-44d14759-d30a-48cf-b0b7-b0d093aacfb3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.820076 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee5a24e009543a38e66a262fae177b9', 'request_id': 'req-44d14759-d30a-48cf-b0b7-b0d093aacfb3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.827660 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee5a24e009543a38e66a262fae177b9', 'request_id': 'req-44d14759-d30a-48cf-b0b7-b0d093aacfb3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.843626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] The token cache is being invalidate because project 0ee5a24e009543a38e66a262fae177b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:35.843950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.844729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-44d14759-d30a-48cf-b0b7-b0d093aacfb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.846722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1388/2758] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:35 2022] DELETE /identity/v3/projects/0ee5a24e009543a38e66a262fae177b9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:35.849071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73c1eb31d0ac48a4b0adb5bc725cd186', 'request_id': 'req-adf91196-49e8-4222-b69a-98e7568976c6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.853507 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556f27a-5736-4e2d-9c7f-c08d129996af None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:35.853709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556f27a-5736-4e2d-9c7f-c08d129996af None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:35.854019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556f27a-5736-4e2d-9c7f-c08d129996af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:35.856694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73c1eb31d0ac48a4b0adb5bc725cd186', 'request_id': 'req-adf91196-49e8-4222-b69a-98e7568976c6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.863731 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73c1eb31d0ac48a4b0adb5bc725cd186', 'request_id': 'req-adf91196-49e8-4222-b69a-98e7568976c6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.877503 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4556f27a-5736-4e2d-9c7f-c08d129996af None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:35.878770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4556f27a-5736-4e2d-9c7f-c08d129996af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:35.882351 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] The token cache is being invalidate because project 73c1eb31d0ac48a4b0adb5bc725cd186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:35.882649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.883309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-adf91196-49e8-4222-b69a-98e7568976c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:35.888389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1371/2759] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:35 2022] DELETE /identity/v3/projects/73c1eb31d0ac48a4b0adb5bc725cd186 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:35.889780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f81734-599f-494d-b431-98df901cd5fa None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:35.890090 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f81734-599f-494d-b431-98df901cd5fa None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:35.890353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f81734-599f-494d-b431-98df901cd5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:35.911618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b4f81734-599f-494d-b431-98df901cd5fa None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:35.913043 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b4f81734-599f-494d-b431-98df901cd5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:35.919259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:35.940542 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:35.956294 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1389/2760] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:35.964327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57434c-7030-4ab5-bc43-3bb7093f5ba3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:35.964680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57434c-7030-4ab5-bc43-3bb7093f5ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:35.965023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57434c-7030-4ab5-bc43-3bb7093f5ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:35.987767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fa57434c-7030-4ab5-bc43-3bb7093f5ba3 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:35.989460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fa57434c-7030-4ab5-bc43-3bb7093f5ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:35.994780 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1372/2761] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:36.003303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c0828-3139-4b1e-bd9a-ddf8ce48ed12 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.003684 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c0828-3139-4b1e-bd9a-ddf8ce48ed12 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.003972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c0828-3139-4b1e-bd9a-ddf8ce48ed12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.009784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1390/2762] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:36.014098 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57434c-7030-4ab5-bc43-3bb7093f5ba3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.025813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57434c-7030-4ab5-bc43-3bb7093f5ba3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.032479 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63627b11-fdec-497e-93e7-29a953792876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.033225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63627b11-fdec-497e-93e7-29a953792876 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.033566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63627b11-fdec-497e-93e7-29a953792876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.033854 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63627b11-fdec-497e-93e7-29a953792876 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.034282 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a04c0828-3139-4b1e-bd9a-ddf8ce48ed12 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:36.034549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63627b11-fdec-497e-93e7-29a953792876 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:55:36.034853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63627b11-fdec-497e-93e7-29a953792876 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.035636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a04c0828-3139-4b1e-bd9a-ddf8ce48ed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:36.037171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63627b11-fdec-497e-93e7-29a953792876 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.048008 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1373/2763] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:36.051846 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c0828-3139-4b1e-bd9a-ddf8ce48ed12 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.053088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1391/2764] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:55:36.059033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63627b11-fdec-497e-93e7-29a953792876 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.062073 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c0828-3139-4b1e-bd9a-ddf8ce48ed12 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.068566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafb0de-d82a-4176-b574-a1c718689a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.070038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafb0de-d82a-4176-b574-a1c718689a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.070038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafb0de-d82a-4176-b574-a1c718689a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.070038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafb0de-d82a-4176-b574-a1c718689a97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.070555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafb0de-d82a-4176-b574-a1c718689a97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:55:36.070866 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafb0de-d82a-4176-b574-a1c718689a97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.070981 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63627b11-fdec-497e-93e7-29a953792876 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.073158 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafb0de-d82a-4176-b574-a1c718689a97 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.077683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f2afc-d81a-48d1-9b9d-339545b14b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.078448 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f2afc-d81a-48d1-9b9d-339545b14b1e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.078799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f2afc-d81a-48d1-9b9d-339545b14b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.079142 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f2afc-d81a-48d1-9b9d-339545b14b1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.080023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f2afc-d81a-48d1-9b9d-339545b14b1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.082153 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f2afc-d81a-48d1-9b9d-339545b14b1e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.091600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1374/2765] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:55:36.093147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafb0de-d82a-4176-b574-a1c718689a97 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.104040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafb0de-d82a-4176-b574-a1c718689a97 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.109196 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaa5a3-eec2-4bee-b611-96339986e2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.109799 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaa5a3-eec2-4bee-b611-96339986e2ac admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.110082 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaa5a3-eec2-4bee-b611-96339986e2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.110362 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaa5a3-eec2-4bee-b611-96339986e2ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.111040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdaa5a3-eec2-4bee-b611-96339986e2ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.112993 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdaa5a3-eec2-4bee-b611-96339986e2ac admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.124548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1392/2766] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:36 2022] POST /identity/v3/projects => generated 397 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:55:36.126191 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f2afc-d81a-48d1-9b9d-339545b14b1e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.146854 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1375/2767] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:36 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:55:36.150997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaa5a3-eec2-4bee-b611-96339986e2ac admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.195446 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f2afc-d81a-48d1-9b9d-339545b14b1e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.217841 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaa5a3-eec2-4bee-b611-96339986e2ac admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.222726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9a954-1090-4aa8-ad64-76728df0a7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.223455 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9a954-1090-4aa8-ad64-76728df0a7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.223962 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9a954-1090-4aa8-ad64-76728df0a7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.224245 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9a954-1090-4aa8-ad64-76728df0a7f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.224946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9a954-1090-4aa8-ad64-76728df0a7f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.226878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9a954-1090-4aa8-ad64-76728df0a7f5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.252810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1376/2768] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:36 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:36.254119 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9a954-1090-4aa8-ad64-76728df0a7f5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.260759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655332-d833-4b01-95e8-8e0a1e396351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.261431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655332-d833-4b01-95e8-8e0a1e396351 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.261715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655332-d833-4b01-95e8-8e0a1e396351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.261989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655332-d833-4b01-95e8-8e0a1e396351 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.262759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24655332-d833-4b01-95e8-8e0a1e396351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.264969 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24655332-d833-4b01-95e8-8e0a1e396351 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.265227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9a954-1090-4aa8-ad64-76728df0a7f5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.270051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fa6db-3fb5-4279-afbc-f1640bb464f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.271353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fa6db-3fb5-4279-afbc-f1640bb464f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.271688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fa6db-3fb5-4279-afbc-f1640bb464f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.271897 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fa6db-3fb5-4279-afbc-f1640bb464f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.272566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17fa6db-3fb5-4279-afbc-f1640bb464f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.274207 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17fa6db-3fb5-4279-afbc-f1640bb464f6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.285209 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1377/2769] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:55:36.290167 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fa6db-3fb5-4279-afbc-f1640bb464f6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.292415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1393/2770] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:36 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:36.296470 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655332-d833-4b01-95e8-8e0a1e396351 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.300229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fa6db-3fb5-4279-afbc-f1640bb464f6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.306577 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067ebc9-a9df-4c43-929f-7aec5cb1971d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.307264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067ebc9-a9df-4c43-929f-7aec5cb1971d admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.307622 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067ebc9-a9df-4c43-929f-7aec5cb1971d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.307725 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655332-d833-4b01-95e8-8e0a1e396351 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.308013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067ebc9-a9df-4c43-929f-7aec5cb1971d admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.308694 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067ebc9-a9df-4c43-929f-7aec5cb1971d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.310337 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067ebc9-a9df-4c43-929f-7aec5cb1971d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.313693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388445dc-9a01-40ef-953d-d7e46a1c5104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.314509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388445dc-9a01-40ef-953d-d7e46a1c5104 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.314892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388445dc-9a01-40ef-953d-d7e46a1c5104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.315337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388445dc-9a01-40ef-953d-d7e46a1c5104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.316085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388445dc-9a01-40ef-953d-d7e46a1c5104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.318894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388445dc-9a01-40ef-953d-d7e46a1c5104 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.322595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1378/2771] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:55:36.326958 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067ebc9-a9df-4c43-929f-7aec5cb1971d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.334886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1394/2772] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:36.338968 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388445dc-9a01-40ef-953d-d7e46a1c5104 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.341406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067ebc9-a9df-4c43-929f-7aec5cb1971d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.348551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596c990-6a81-4acd-9250-6651842b2d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.349615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c990-6a81-4acd-9250-6651842b2d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.349948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c990-6a81-4acd-9250-6651842b2d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.350409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c990-6a81-4acd-9250-6651842b2d6d admin admin] PATH_INFO: `/v3/projects/27e76ec2aef647a282b7154dd8ebb33f/users/0df11267623442fa976dcab89eedd4d8/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.355400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388445dc-9a01-40ef-953d-d7e46a1c5104 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.362376 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977abe5e-9308-4a45-abd3-64530b4ca180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.363307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abe5e-9308-4a45-abd3-64530b4ca180 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.363720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abe5e-9308-4a45-abd3-64530b4ca180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.364847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abe5e-9308-4a45-abd3-64530b4ca180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.365914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977abe5e-9308-4a45-abd3-64530b4ca180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.368715 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977abe5e-9308-4a45-abd3-64530b4ca180 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.374949 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596c990-6a81-4acd-9250-6651842b2d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e76ec2aef647a282b7154dd8ebb33f, user_id=0df11267623442fa976dcab89eedd4d8, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.380682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596c990-6a81-4acd-9250-6651842b2d6d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.386986 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1395/2773] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:55:36.391577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977abe5e-9308-4a45-abd3-64530b4ca180 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.397077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1379/2774] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:36 2022] PUT /identity/v3/projects/27e76ec2aef647a282b7154dd8ebb33f/users/0df11267623442fa976dcab89eedd4d8/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:55:36.405210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977abe5e-9308-4a45-abd3-64530b4ca180 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.439628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:36.465743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7522c-1b20-4fbb-b185-57f4e432c0c8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.465941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7522c-1b20-4fbb-b185-57f4e432c0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.466204 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7522c-1b20-4fbb-b185-57f4e432c0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.473136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350429d9-c89c-4dde-aea2-f389dbcde37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.474358 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350429d9-c89c-4dde-aea2-f389dbcde37a admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.474596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350429d9-c89c-4dde-aea2-f389dbcde37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.474872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350429d9-c89c-4dde-aea2-f389dbcde37a admin admin] PATH_INFO: `/v3/projects/642857bcba2f48ce81af4b7d7ecaade3/users/ab5bee4f3cad4238b85c913780a5bc7a/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.491161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350429d9-c89c-4dde-aea2-f389dbcde37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=642857bcba2f48ce81af4b7d7ecaade3, user_id=ab5bee4f3cad4238b85c913780a5bc7a, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.493934 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350429d9-c89c-4dde-aea2-f389dbcde37a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.496405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:36.500440 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-90e7522c-1b20-4fbb-b185-57f4e432c0c8 None None] MFA Rules not processed for user `0df11267623442fa976dcab89eedd4d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:36.502771 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-90e7522c-1b20-4fbb-b185-57f4e432c0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:36.508645 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1396/2775] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:36 2022] PUT /identity/v3/projects/642857bcba2f48ce81af4b7d7ecaade3/users/ab5bee4f3cad4238b85c913780a5bc7a/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:36.519450 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6495f33-f8cf-4d1f-adf0-a92e02b26164 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.519736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6495f33-f8cf-4d1f-adf0-a92e02b26164 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.520008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6495f33-f8cf-4d1f-adf0-a92e02b26164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.549914 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b6495f33-f8cf-4d1f-adf0-a92e02b26164 None None] MFA Rules not processed for user `ab5bee4f3cad4238b85c913780a5bc7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:36.551124 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b6495f33-f8cf-4d1f-adf0-a92e02b26164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:36.572370 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1380/2776] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:36.584670 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7522c-1b20-4fbb-b185-57f4e432c0c8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.586203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-90e7522c-1b20-4fbb-b185-57f4e432c0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:36.622457 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1397/2777] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:36.634167 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6495f33-f8cf-4d1f-adf0-a92e02b26164 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.635554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b6495f33-f8cf-4d1f-adf0-a92e02b26164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:36.645422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7522c-1b20-4fbb-b185-57f4e432c0c8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.647796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6495f33-f8cf-4d1f-adf0-a92e02b26164 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.649224 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bed7f-6843-42b2-aea0-157818325e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.649829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bed7f-6843-42b2-aea0-157818325e54 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.650107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bed7f-6843-42b2-aea0-157818325e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.650379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bed7f-6843-42b2-aea0-157818325e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.651549 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742f746-7eec-43f6-b552-bbe8ba3e93d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.652152 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f746-7eec-43f6-b552-bbe8ba3e93d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.652432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f746-7eec-43f6-b552-bbe8ba3e93d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.652734 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f746-7eec-43f6-b552-bbe8ba3e93d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.658996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8bed7f-6843-42b2-aea0-157818325e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.660423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8bed7f-6843-42b2-aea0-157818325e54 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.662074 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742f746-7eec-43f6-b552-bbe8ba3e93d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.663547 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742f746-7eec-43f6-b552-bbe8ba3e93d6 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.670040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1381/2778] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:55:36.676330 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1398/2779] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:36 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:55:36.696972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bed7f-6843-42b2-aea0-157818325e54 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.767559 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bed7f-6843-42b2-aea0-157818325e54 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:36.772425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:36.773291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.773403 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.773733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] PATH_INFO: `/v3/users/0df11267623442fa976dcab89eedd4d8` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.774769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] RBAC: Authorizing `identity:delete_user(user_id=0df11267623442fa976dcab89eedd4d8)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:36.777450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:36.836088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0df11267623442fa976dcab89eedd4d8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:36.836919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0df11267623442fa976dcab89eedd4d8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:36.839493 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:36.845594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0df11267623442fa976dcab89eedd4d8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:36.855160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0df11267623442fa976dcab89eedd4d8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:36.855946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0df11267623442fa976dcab89eedd4d8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:36.877627 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1382/2780] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:36 2022] DELETE /identity/v3/users/0df11267623442fa976dcab89eedd4d8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:36.882523 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c343-d53b-4381-8a0d-aba4c304e218 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:36.882781 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c343-d53b-4381-8a0d-aba4c304e218 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:36.883059 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c343-d53b-4381-8a0d-aba4c304e218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:36.903893 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a9e1c343-d53b-4381-8a0d-aba4c304e218 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:36.905136 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a9e1c343-d53b-4381-8a0d-aba4c304e218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:36.970824 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1399/2781] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:36.982029 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:36.983758 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:37.048585 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3dbc7-7454-497b-95b9-ee2f0e3e8ace admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:37.053314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bf9b4-e4d0-45a8-9ff6-50be5e599982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:37.053965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bf9b4-e4d0-45a8-9ff6-50be5e599982 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:37.054282 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bf9b4-e4d0-45a8-9ff6-50be5e599982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:37.054622 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bf9b4-e4d0-45a8-9ff6-50be5e599982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:37.064287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bf9b4-e4d0-45a8-9ff6-50be5e599982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:37.065946 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bf9b4-e4d0-45a8-9ff6-50be5e599982 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:37.077465 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1383/2782] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:55:37.416255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1c343-d53b-4381-8a0d-aba4c304e218 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:37.418148 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a9e1c343-d53b-4381-8a0d-aba4c304e218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:37.435393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1c343-d53b-4381-8a0d-aba4c304e218 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:37.440963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:37.441468 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:37.442504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:37.442787 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] PATH_INFO: `/v3/projects/27e76ec2aef647a282b7154dd8ebb33f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:37.444305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e76ec2aef647a282b7154dd8ebb33f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:37.446145 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:37.478340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:37.497301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e76ec2aef647a282b7154dd8ebb33f', 'request_id': 'req-ea0437d3-95cd-4626-ba3c-1809ecdd138a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:37.503459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e76ec2aef647a282b7154dd8ebb33f', 'request_id': 'req-ea0437d3-95cd-4626-ba3c-1809ecdd138a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:37.509574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e76ec2aef647a282b7154dd8ebb33f', 'request_id': 'req-ea0437d3-95cd-4626-ba3c-1809ecdd138a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:37.522834 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] The token cache is being invalidate because project 27e76ec2aef647a282b7154dd8ebb33f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:37.523128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:37.523792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ea0437d3-95cd-4626-ba3c-1809ecdd138a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:37.525366 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1400/2783] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:37 2022] DELETE /identity/v3/projects/27e76ec2aef647a282b7154dd8ebb33f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:37.530394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf580-3c69-4c08-9853-4d77bd465b22 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:37.530626 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf580-3c69-4c08-9853-4d77bd465b22 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:37.530904 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf580-3c69-4c08-9853-4d77bd465b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:37.550409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1f9cf580-3c69-4c08-9853-4d77bd465b22 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:37.551744 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1f9cf580-3c69-4c08-9853-4d77bd465b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:37.611810 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1384/2784] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:37.620220 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf005-6a43-4348-8afa-4a2d1e742183 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:37.620582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf005-6a43-4348-8afa-4a2d1e742183 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:37.620889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf005-6a43-4348-8afa-4a2d1e742183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:37.643310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-725bf005-6a43-4348-8afa-4a2d1e742183 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:37.644676 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-725bf005-6a43-4348-8afa-4a2d1e742183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:37.653767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1401/2785] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:37.658521 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cf580-3c69-4c08-9853-4d77bd465b22 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:37.669798 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cf580-3c69-4c08-9853-4d77bd465b22 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:37.679485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480916-5289-4d88-bb03-35a41f1c5438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:37.679485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480916-5289-4d88-bb03-35a41f1c5438 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:37.679485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480916-5289-4d88-bb03-35a41f1c5438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:37.679485 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480916-5289-4d88-bb03-35a41f1c5438 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:37.679928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0480916-5289-4d88-bb03-35a41f1c5438 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:55:37.679928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0480916-5289-4d88-bb03-35a41f1c5438 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:37.679928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0480916-5289-4d88-bb03-35a41f1c5438 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:37.693475 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1385/2786] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:55:37.698231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bf005-6a43-4348-8afa-4a2d1e742183 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:37.707722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bf005-6a43-4348-8afa-4a2d1e742183 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:37.712699 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a0901-add6-4063-8646-1f81451363e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:37.713296 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a0901-add6-4063-8646-1f81451363e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:37.713575 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a0901-add6-4063-8646-1f81451363e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:37.713852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a0901-add6-4063-8646-1f81451363e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:37.714521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a0901-add6-4063-8646-1f81451363e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:37.716417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a0901-add6-4063-8646-1f81451363e5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:37.745177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1402/2787] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:37 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:37.749287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480916-5289-4d88-bb03-35a41f1c5438 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:37.761302 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480916-5289-4d88-bb03-35a41f1c5438 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:37.816791 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:37.826314 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fcd08-2e3a-45ad-b33d-2d70554f9c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:37.827211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fcd08-2e3a-45ad-b33d-2d70554f9c9e admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:37.827733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fcd08-2e3a-45ad-b33d-2d70554f9c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:37.828039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fcd08-2e3a-45ad-b33d-2d70554f9c9e admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:37.828795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fcd08-2e3a-45ad-b33d-2d70554f9c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:37.830927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fcd08-2e3a-45ad-b33d-2d70554f9c9e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:37.854458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1386/2788] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:37 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:37.858025 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a0901-add6-4063-8646-1f81451363e5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:37.870671 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a0901-add6-4063-8646-1f81451363e5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:37.875596 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd6f77-427c-4a7a-b437-a4ce26139e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:37.876246 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd6f77-427c-4a7a-b437-a4ce26139e3a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:37.876566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd6f77-427c-4a7a-b437-a4ce26139e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:37.876892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd6f77-427c-4a7a-b437-a4ce26139e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:37.877627 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd6f77-427c-4a7a-b437-a4ce26139e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:37.879386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd6f77-427c-4a7a-b437-a4ce26139e3a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:37.891766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1403/2789] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:37.896112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fcd08-2e3a-45ad-b33d-2d70554f9c9e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:37.907611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fcd08-2e3a-45ad-b33d-2d70554f9c9e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:37.912874 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a724fc8-30df-48c9-b1b5-a2cdcb6227f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:37.913584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a724fc8-30df-48c9-b1b5-a2cdcb6227f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:37.913896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a724fc8-30df-48c9-b1b5-a2cdcb6227f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:37.914211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a724fc8-30df-48c9-b1b5-a2cdcb6227f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:37.914952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a724fc8-30df-48c9-b1b5-a2cdcb6227f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:37.916941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a724fc8-30df-48c9-b1b5-a2cdcb6227f2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:37.929610 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1387/2790] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:37.934328 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd6f77-427c-4a7a-b437-a4ce26139e3a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:37.946979 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd6f77-427c-4a7a-b437-a4ce26139e3a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:37.952169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7f53f-95c8-4ec0-a0e6-0c01c3b35ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:37.952941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7f53f-95c8-4ec0-a0e6-0c01c3b35ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:37.953285 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7f53f-95c8-4ec0-a0e6-0c01c3b35ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:37.953557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7f53f-95c8-4ec0-a0e6-0c01c3b35ba4 admin admin] PATH_INFO: `/v3/projects/881fc8c528cd40a3b5dbf4c49b467b14/users/bc731aa0770745ebadf4cf0d4a3114be/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:37.969577 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db7f53f-95c8-4ec0-a0e6-0c01c3b35ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=881fc8c528cd40a3b5dbf4c49b467b14, user_id=bc731aa0770745ebadf4cf0d4a3114be, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:37.972247 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db7f53f-95c8-4ec0-a0e6-0c01c3b35ba4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:37.986710 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1404/2791] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:37 2022] PUT /identity/v3/projects/881fc8c528cd40a3b5dbf4c49b467b14/users/bc731aa0770745ebadf4cf0d4a3114be/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:55:37.997921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2212433-87f7-44a8-9cad-67f352c666e9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:37.998208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2212433-87f7-44a8-9cad-67f352c666e9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:37.998481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2212433-87f7-44a8-9cad-67f352c666e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.026441 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e2212433-87f7-44a8-9cad-67f352c666e9 None None] MFA Rules not processed for user `bc731aa0770745ebadf4cf0d4a3114be`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:38.027686 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e2212433-87f7-44a8-9cad-67f352c666e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:38.093114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1388/2792] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:38.097693 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7f53f-95c8-4ec0-a0e6-0c01c3b35ba4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:38.162178 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7f53f-95c8-4ec0-a0e6-0c01c3b35ba4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:38.166995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06705ed8-2e1a-4fa1-83ef-f20b5b51ea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:38.167611 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06705ed8-2e1a-4fa1-83ef-f20b5b51ea79 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.167894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06705ed8-2e1a-4fa1-83ef-f20b5b51ea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.168176 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06705ed8-2e1a-4fa1-83ef-f20b5b51ea79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.168861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06705ed8-2e1a-4fa1-83ef-f20b5b51ea79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:38.170816 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06705ed8-2e1a-4fa1-83ef-f20b5b51ea79 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:38.202717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1405/2793] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:38 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:38.203828 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2212433-87f7-44a8-9cad-67f352c666e9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:38.220591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2212433-87f7-44a8-9cad-67f352c666e9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:38.278458 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9df04-dda5-4cf7-b150-171fa9a9f070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:38.279065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9df04-dda5-4cf7-b150-171fa9a9f070 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.279348 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9df04-dda5-4cf7-b150-171fa9a9f070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.279619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9df04-dda5-4cf7-b150-171fa9a9f070 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.280261 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9df04-dda5-4cf7-b150-171fa9a9f070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:38.282216 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9df04-dda5-4cf7-b150-171fa9a9f070 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:38.304267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1389/2794] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:38 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:38.308922 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06705ed8-2e1a-4fa1-83ef-f20b5b51ea79 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:38.320006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06705ed8-2e1a-4fa1-83ef-f20b5b51ea79 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:38.324709 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88117867-5f15-4c39-962c-57cbbf8cec3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:38.325305 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88117867-5f15-4c39-962c-57cbbf8cec3f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.325585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88117867-5f15-4c39-962c-57cbbf8cec3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.325855 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88117867-5f15-4c39-962c-57cbbf8cec3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.326512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88117867-5f15-4c39-962c-57cbbf8cec3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:38.328081 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88117867-5f15-4c39-962c-57cbbf8cec3f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:38.339474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1406/2795] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:38.343776 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9df04-dda5-4cf7-b150-171fa9a9f070 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:38.354405 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9df04-dda5-4cf7-b150-171fa9a9f070 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:38.359131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb6183-2538-4c7d-a3fb-8cda3280021e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:38.359730 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb6183-2538-4c7d-a3fb-8cda3280021e admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.360012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb6183-2538-4c7d-a3fb-8cda3280021e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.360287 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb6183-2538-4c7d-a3fb-8cda3280021e admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.360975 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb6183-2538-4c7d-a3fb-8cda3280021e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:38.362569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb6183-2538-4c7d-a3fb-8cda3280021e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:38.373085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1390/2796] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:55:38.377385 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88117867-5f15-4c39-962c-57cbbf8cec3f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:38.388864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88117867-5f15-4c39-962c-57cbbf8cec3f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:38.393658 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb8747-b10d-4661-b86c-11a14d39e9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:38.394308 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8747-b10d-4661-b86c-11a14d39e9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.394592 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8747-b10d-4661-b86c-11a14d39e9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.394870 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8747-b10d-4661-b86c-11a14d39e9e1 admin admin] PATH_INFO: `/v3/projects/f32a1cd0a01e40d1a42cbf3d780f3012/users/da7cdf3b780744a997811b881ece0b9c/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.415024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb8747-b10d-4661-b86c-11a14d39e9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f32a1cd0a01e40d1a42cbf3d780f3012, user_id=da7cdf3b780744a997811b881ece0b9c, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:38.417842 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb8747-b10d-4661-b86c-11a14d39e9e1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:38.431405 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:38.436932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1407/2797] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:38 2022] PUT /identity/v3/projects/f32a1cd0a01e40d1a42cbf3d780f3012/users/da7cdf3b780744a997811b881ece0b9c/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:38.451041 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ae55f-6d64-4830-a9e9-0c5b085c69cf None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.451393 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ae55f-6d64-4830-a9e9-0c5b085c69cf None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.451711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ae55f-6d64-4830-a9e9-0c5b085c69cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.481489 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-246ae55f-6d64-4830-a9e9-0c5b085c69cf None None] MFA Rules not processed for user `da7cdf3b780744a997811b881ece0b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:38.482765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-246ae55f-6d64-4830-a9e9-0c5b085c69cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:38.516387 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:38.555263 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1391/2798] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:38.559964 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb8747-b10d-4661-b86c-11a14d39e9e1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:38.623641 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb8747-b10d-4661-b86c-11a14d39e9e1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:38.628315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8facfb-a93e-4e47-8954-9746cb63fa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:38.628945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8facfb-a93e-4e47-8954-9746cb63fa6f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.629234 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8facfb-a93e-4e47-8954-9746cb63fa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.629795 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8facfb-a93e-4e47-8954-9746cb63fa6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.630659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8facfb-a93e-4e47-8954-9746cb63fa6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:38.632587 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8facfb-a93e-4e47-8954-9746cb63fa6f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:38.659807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1408/2799] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:38 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:38.664965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ae55f-6d64-4830-a9e9-0c5b085c69cf None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:38.675621 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ae55f-6d64-4830-a9e9-0c5b085c69cf None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:38.738487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80eafaa-5a34-4e66-ac28-dc342a070278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:38.739941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80eafaa-5a34-4e66-ac28-dc342a070278 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.740228 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80eafaa-5a34-4e66-ac28-dc342a070278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.740536 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80eafaa-5a34-4e66-ac28-dc342a070278 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.741188 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80eafaa-5a34-4e66-ac28-dc342a070278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:38.743109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80eafaa-5a34-4e66-ac28-dc342a070278 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:38.766889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1392/2800] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:38 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:38.771894 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8facfb-a93e-4e47-8954-9746cb63fa6f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:38.786341 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8facfb-a93e-4e47-8954-9746cb63fa6f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:38.792915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91645d-21a6-4883-a613-05dd8ba62311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:38.793524 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91645d-21a6-4883-a613-05dd8ba62311 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.793805 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91645d-21a6-4883-a613-05dd8ba62311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.794084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91645d-21a6-4883-a613-05dd8ba62311 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.794730 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91645d-21a6-4883-a613-05dd8ba62311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:38.796327 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91645d-21a6-4883-a613-05dd8ba62311 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:38.809286 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1409/2801] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:38.813649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80eafaa-5a34-4e66-ac28-dc342a070278 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:38.824122 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80eafaa-5a34-4e66-ac28-dc342a070278 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:38.828806 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910222fb-5a68-4dfe-a5ff-7bd4870dac01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:38.829460 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910222fb-5a68-4dfe-a5ff-7bd4870dac01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.829742 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910222fb-5a68-4dfe-a5ff-7bd4870dac01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.830017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910222fb-5a68-4dfe-a5ff-7bd4870dac01 admin admin] PATH_INFO: `/v3/projects/8c97413bfeec4aa589b380103c705b29/users/248ae23b98114609bdd5701f27f49dde/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.845245 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910222fb-5a68-4dfe-a5ff-7bd4870dac01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c97413bfeec4aa589b380103c705b29, user_id=248ae23b98114609bdd5701f27f49dde, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:38.847883 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910222fb-5a68-4dfe-a5ff-7bd4870dac01 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:38.861311 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1393/2802] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:38 2022] PUT /identity/v3/projects/8c97413bfeec4aa589b380103c705b29/users/248ae23b98114609bdd5701f27f49dde/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:38.865962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91645d-21a6-4883-a613-05dd8ba62311 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:38.875798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91645d-21a6-4883-a613-05dd8ba62311 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:38.932641 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adb2ed-f7e6-4c6b-bfd3-1863e700070a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:38.933261 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adb2ed-f7e6-4c6b-bfd3-1863e700070a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.933536 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adb2ed-f7e6-4c6b-bfd3-1863e700070a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.933811 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adb2ed-f7e6-4c6b-bfd3-1863e700070a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.934453 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0adb2ed-f7e6-4c6b-bfd3-1863e700070a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:38.936040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0adb2ed-f7e6-4c6b-bfd3-1863e700070a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:38.947045 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1410/2803] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:38.951225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910222fb-5a68-4dfe-a5ff-7bd4870dac01 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:38.961578 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910222fb-5a68-4dfe-a5ff-7bd4870dac01 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:38.966210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ca09f-cfa5-426c-aabf-4021f32df4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:38.966853 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ca09f-cfa5-426c-aabf-4021f32df4eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.967136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ca09f-cfa5-426c-aabf-4021f32df4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.967409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ca09f-cfa5-426c-aabf-4021f32df4eb admin admin] PATH_INFO: `/v3/projects/8c97413bfeec4aa589b380103c705b29/users/248ae23b98114609bdd5701f27f49dde/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:38.969358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ca09f-cfa5-426c-aabf-4021f32df4eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c97413bfeec4aa589b380103c705b29, user_id=248ae23b98114609bdd5701f27f49dde, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:38.971976 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ca09f-cfa5-426c-aabf-4021f32df4eb admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:38.982910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1394/2804] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:38 2022] PUT /identity/v3/projects/8c97413bfeec4aa589b380103c705b29/users/248ae23b98114609bdd5701f27f49dde/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:38.994199 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdefeed-e078-4b4a-9c69-c675c43a7ab7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:38.994464 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdefeed-e078-4b4a-9c69-c675c43a7ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:38.994738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdefeed-e078-4b4a-9c69-c675c43a7ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:39.023200 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5bdefeed-e078-4b4a-9c69-c675c43a7ab7 None None] MFA Rules not processed for user `248ae23b98114609bdd5701f27f49dde`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:39.024416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5bdefeed-e078-4b4a-9c69-c675c43a7ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:39.087649 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:39.098065 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1411/2805] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:39.110259 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ca09f-cfa5-426c-aabf-4021f32df4eb admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:39.112177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7b8ca09f-cfa5-426c-aabf-4021f32df4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:39.170751 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ca09f-cfa5-426c-aabf-4021f32df4eb admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:39.174729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da4db0-2575-452c-9aa7-d1731e50c95d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:39.175417 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da4db0-2575-452c-9aa7-d1731e50c95d service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:39.175728 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da4db0-2575-452c-9aa7-d1731e50c95d service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:39.176006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da4db0-2575-452c-9aa7-d1731e50c95d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:39.217564 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:39.237258 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da4db0-2575-452c-9aa7-d1731e50c95d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:39.238767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da4db0-2575-452c-9aa7-d1731e50c95d service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:39.248025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1395/2806] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:42.923476 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdefeed-e078-4b4a-9c69-c675c43a7ab7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:42.934187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdefeed-e078-4b4a-9c69-c675c43a7ab7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:42.937796 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5435d2f-6cc4-4f64-87de-98b299d533be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:42.938392 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5435d2f-6cc4-4f64-87de-98b299d533be service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:42.938675 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5435d2f-6cc4-4f64-87de-98b299d533be service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:42.938950 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5435d2f-6cc4-4f64-87de-98b299d533be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:43.002007 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5435d2f-6cc4-4f64-87de-98b299d533be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:43.003646 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5435d2f-6cc4-4f64-87de-98b299d533be service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:43.013355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1412/2807] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:43.573012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da4db0-2575-452c-9aa7-d1731e50c95d service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:43.581773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da4db0-2575-452c-9aa7-d1731e50c95d service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:43.586021 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d394f01-daf6-42cd-8661-61979cf51ebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:43.586613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d394f01-daf6-42cd-8661-61979cf51ebd service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:43.586887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d394f01-daf6-42cd-8661-61979cf51ebd service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:43.587160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d394f01-daf6-42cd-8661-61979cf51ebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:43.596055 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d394f01-daf6-42cd-8661-61979cf51ebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:43.597496 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d394f01-daf6-42cd-8661-61979cf51ebd service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:43.607051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1396/2808] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:53.009415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=30 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:53.215570 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5435d2f-6cc4-4f64-87de-98b299d533be service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:53.225891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b5435d2f-6cc4-4f64-87de-98b299d533be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:53.345747 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5435d2f-6cc4-4f64-87de-98b299d533be service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:53.353360 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:53.354477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:53.354936 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:53.355494 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] PATH_INFO: `/v3/users/ab5bee4f3cad4238b85c913780a5bc7a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:53.357484 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab5bee4f3cad4238b85c913780a5bc7a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:53.361185 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:53.432212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab5bee4f3cad4238b85c913780a5bc7a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:53.440311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab5bee4f3cad4238b85c913780a5bc7a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:53.441782 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab5bee4f3cad4238b85c913780a5bc7a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:53.442619 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab5bee4f3cad4238b85c913780a5bc7a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:53.452906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab5bee4f3cad4238b85c913780a5bc7a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:53.481357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1413/2809] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:53 2022] DELETE /identity/v3/users/ab5bee4f3cad4238b85c913780a5bc7a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:53.487767 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8056a0-c7c6-4a3e-8311-167ca72574ab None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:53.488574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8056a0-c7c6-4a3e-8311-167ca72574ab None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:53.488864 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8056a0-c7c6-4a3e-8311-167ca72574ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:53.514602 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9f8056a0-c7c6-4a3e-8311-167ca72574ab None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:53.516940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9f8056a0-c7c6-4a3e-8311-167ca72574ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:53.587354 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1397/2810] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:53.597779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:53.599582 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:53.668315 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bc7c4-f353-400f-83b6-1ca3affdf6c3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:53.671960 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104aaa3d-988a-418f-811e-feebf03b8d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:53.672601 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104aaa3d-988a-418f-811e-feebf03b8d6a service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:53.672882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104aaa3d-988a-418f-811e-feebf03b8d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:53.673150 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104aaa3d-988a-418f-811e-feebf03b8d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:53.690999 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104aaa3d-988a-418f-811e-feebf03b8d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:53.693433 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104aaa3d-988a-418f-811e-feebf03b8d6a service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:53.712799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1414/2811] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:55:54.063708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8056a0-c7c6-4a3e-8311-167ca72574ab None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:54.065089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9f8056a0-c7c6-4a3e-8311-167ca72574ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:54.082974 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8056a0-c7c6-4a3e-8311-167ca72574ab None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:54.088052 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:54.088836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:54.089078 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:54.089389 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] PATH_INFO: `/v3/projects/642857bcba2f48ce81af4b7d7ecaade3` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:54.090657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] RBAC: Authorizing `identity:delete_project(project_id=642857bcba2f48ce81af4b7d7ecaade3)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:54.092616 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:54.148846 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642857bcba2f48ce81af4b7d7ecaade3', 'request_id': 'req-3fa5a51f-2060-45d0-94ff-a5491923e53c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:54.154271 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642857bcba2f48ce81af4b7d7ecaade3', 'request_id': 'req-3fa5a51f-2060-45d0-94ff-a5491923e53c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:54.161226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642857bcba2f48ce81af4b7d7ecaade3', 'request_id': 'req-3fa5a51f-2060-45d0-94ff-a5491923e53c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:54.173708 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] The token cache is being invalidate because project 642857bcba2f48ce81af4b7d7ecaade3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:54.174002 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:54.174726 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3fa5a51f-2060-45d0-94ff-a5491923e53c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:54.177080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1398/2812] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:54 2022] DELETE /identity/v3/projects/642857bcba2f48ce81af4b7d7ecaade3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:54.182102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83232054-664c-48d7-80d2-652ac08bc6b9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:54.182615 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83232054-664c-48d7-80d2-652ac08bc6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:54.182737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83232054-664c-48d7-80d2-652ac08bc6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:54.204512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-83232054-664c-48d7-80d2-652ac08bc6b9 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:54.206100 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-83232054-664c-48d7-80d2-652ac08bc6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:54.226045 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:54.273249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1415/2813] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:54.281123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df385c-203e-418f-b0a3-3752eae598e5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:54.281416 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df385c-203e-418f-b0a3-3752eae598e5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:54.281693 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df385c-203e-418f-b0a3-3752eae598e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:54.304794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d1df385c-203e-418f-b0a3-3752eae598e5 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:54.306072 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d1df385c-203e-418f-b0a3-3752eae598e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:54.320009 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1399/2814] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:54.324281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83232054-664c-48d7-80d2-652ac08bc6b9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:54.334186 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83232054-664c-48d7-80d2-652ac08bc6b9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:54.338735 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397267e-b210-43dc-b313-a4814d2f9eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:54.339386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397267e-b210-43dc-b313-a4814d2f9eaa admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:54.339666 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397267e-b210-43dc-b313-a4814d2f9eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:54.339937 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397267e-b210-43dc-b313-a4814d2f9eaa admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:54.340609 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397267e-b210-43dc-b313-a4814d2f9eaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:55:54.340915 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397267e-b210-43dc-b313-a4814d2f9eaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:54.342494 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397267e-b210-43dc-b313-a4814d2f9eaa admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:54.354540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1416/2815] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:55:54.359016 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df385c-203e-418f-b0a3-3752eae598e5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:54.368959 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df385c-203e-418f-b0a3-3752eae598e5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:54.373551 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741ebab-1b6c-4a41-bbd5-7e84b715f7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:54.374151 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741ebab-1b6c-4a41-bbd5-7e84b715f7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:54.374444 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741ebab-1b6c-4a41-bbd5-7e84b715f7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:54.374735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741ebab-1b6c-4a41-bbd5-7e84b715f7f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:54.375378 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741ebab-1b6c-4a41-bbd5-7e84b715f7f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:54.377358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741ebab-1b6c-4a41-bbd5-7e84b715f7f3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:54.405735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1400/2816] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:54 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:54.410473 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397267e-b210-43dc-b313-a4814d2f9eaa admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:54.420799 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397267e-b210-43dc-b313-a4814d2f9eaa admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:54.490031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d590dc-f3f0-4c98-9200-429753bcf6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:54.490711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d590dc-f3f0-4c98-9200-429753bcf6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:54.491020 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d590dc-f3f0-4c98-9200-429753bcf6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:54.491334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d590dc-f3f0-4c98-9200-429753bcf6a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:54.492057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d590dc-f3f0-4c98-9200-429753bcf6a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:54.494325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d590dc-f3f0-4c98-9200-429753bcf6a9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:54.521531 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1417/2817] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:54 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:55:54.523219 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741ebab-1b6c-4a41-bbd5-7e84b715f7f3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:54.534131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741ebab-1b6c-4a41-bbd5-7e84b715f7f3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:54.539842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8d764-43d5-49d1-9d61-1a4fe7ec985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:54.540514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8d764-43d5-49d1-9d61-1a4fe7ec985a admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:54.540878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8d764-43d5-49d1-9d61-1a4fe7ec985a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:54.541147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8d764-43d5-49d1-9d61-1a4fe7ec985a admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:54.541796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8d764-43d5-49d1-9d61-1a4fe7ec985a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:54.543414 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8d764-43d5-49d1-9d61-1a4fe7ec985a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:54.554358 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1401/2818] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:54.558810 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d590dc-f3f0-4c98-9200-429753bcf6a9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:54.574216 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d590dc-f3f0-4c98-9200-429753bcf6a9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:54.578918 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90155427-72ba-4fbe-ba0c-da2015839aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:54.579539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155427-72ba-4fbe-ba0c-da2015839aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:54.579820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155427-72ba-4fbe-ba0c-da2015839aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:54.580091 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155427-72ba-4fbe-ba0c-da2015839aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:54.580767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90155427-72ba-4fbe-ba0c-da2015839aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:54.583054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90155427-72ba-4fbe-ba0c-da2015839aa8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:54.593179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1418/2819] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:54.600455 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8d764-43d5-49d1-9d61-1a4fe7ec985a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:54.610680 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8d764-43d5-49d1-9d61-1a4fe7ec985a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:54.615942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1a4e6-4981-4fa9-b8b1-94383a672ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:54.616650 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a4e6-4981-4fa9-b8b1-94383a672ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:54.616941 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a4e6-4981-4fa9-b8b1-94383a672ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:54.617249 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a4e6-4981-4fa9-b8b1-94383a672ee9 admin admin] PATH_INFO: `/v3/projects/fe46592aea54410c9f49a0371bb2bc73/users/c6d4923a511342248e34954fc44811c0/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:54.620575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:54.633932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1a4e6-4981-4fa9-b8b1-94383a672ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe46592aea54410c9f49a0371bb2bc73, user_id=c6d4923a511342248e34954fc44811c0, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:54.636743 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1a4e6-4981-4fa9-b8b1-94383a672ee9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:54.650613 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1402/2820] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:54 2022] PUT /identity/v3/projects/fe46592aea54410c9f49a0371bb2bc73/users/c6d4923a511342248e34954fc44811c0/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:54.655825 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046660d-a025-46eb-a4aa-354bac914f9d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:54.656177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046660d-a025-46eb-a4aa-354bac914f9d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:54.656552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046660d-a025-46eb-a4aa-354bac914f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:54.677786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7046660d-a025-46eb-a4aa-354bac914f9d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:54.679106 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7046660d-a025-46eb-a4aa-354bac914f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:54.739906 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1419/2821] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:54.749668 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1a4e6-4981-4fa9-b8b1-94383a672ee9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:54.751397 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0ff1a4e6-4981-4fa9-b8b1-94383a672ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:54.815738 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1a4e6-4981-4fa9-b8b1-94383a672ee9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:54.822824 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a99954-5787-428a-8f56-8578df78fdee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:54.823450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a99954-5787-428a-8f56-8578df78fdee service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:54.823735 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a99954-5787-428a-8f56-8578df78fdee service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:54.824010 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a99954-5787-428a-8f56-8578df78fdee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:54.834947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a99954-5787-428a-8f56-8578df78fdee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:54.836392 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a99954-5787-428a-8f56-8578df78fdee service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:54.846175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1403/2822] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:55.201625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046660d-a025-46eb-a4aa-354bac914f9d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:55.203192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7046660d-a025-46eb-a4aa-354bac914f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:55.217232 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046660d-a025-46eb-a4aa-354bac914f9d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:55.222023 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:55.222849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:55.223204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:55.224223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] PATH_INFO: `/v3/users/bc731aa0770745ebadf4cf0d4a3114be` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:55.225110 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc731aa0770745ebadf4cf0d4a3114be)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:55.227105 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:55.252629 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:55.284580 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc731aa0770745ebadf4cf0d4a3114be'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:55.294432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc731aa0770745ebadf4cf0d4a3114be'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:55.295268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc731aa0770745ebadf4cf0d4a3114be'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:55.296054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc731aa0770745ebadf4cf0d4a3114be'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:55.305163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc731aa0770745ebadf4cf0d4a3114be'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:55.325742 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1420/2823] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:55 2022] DELETE /identity/v3/users/bc731aa0770745ebadf4cf0d4a3114be => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:55.330236 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896d79a-3122-4cc5-9d52-931b1f8443dc None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:55.330524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896d79a-3122-4cc5-9d52-931b1f8443dc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:55.330802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896d79a-3122-4cc5-9d52-931b1f8443dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:55.352247 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8896d79a-3122-4cc5-9d52-931b1f8443dc None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:55.353517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8896d79a-3122-4cc5-9d52-931b1f8443dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:55.417136 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1404/2824] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:55.427708 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:55.429252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:55.499468 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149659d-3ccf-4be5-adf2-54f21711fb74 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:55.503849 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda697b-5c0d-4734-abf5-14286bbc0777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:55.505077 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda697b-5c0d-4734-abf5-14286bbc0777 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:55.505077 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda697b-5c0d-4734-abf5-14286bbc0777 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:55.505298 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda697b-5c0d-4734-abf5-14286bbc0777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:55.516736 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda697b-5c0d-4734-abf5-14286bbc0777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:55.518456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda697b-5c0d-4734-abf5-14286bbc0777 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:55.531574 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1421/2825] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:56.056153 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896d79a-3122-4cc5-9d52-931b1f8443dc None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:56.058101 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8896d79a-3122-4cc5-9d52-931b1f8443dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:56.067224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda697b-5c0d-4734-abf5-14286bbc0777 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:56.068659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bdda697b-5c0d-4734-abf5-14286bbc0777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:56.074667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896d79a-3122-4cc5-9d52-931b1f8443dc None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:56.081309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:56.082693 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:56.083179 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:56.083676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] PATH_INFO: `/v3/projects/881fc8c528cd40a3b5dbf4c49b467b14` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:56.084973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=881fc8c528cd40a3b5dbf4c49b467b14)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:56.087532 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda697b-5c0d-4734-abf5-14286bbc0777 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:56.087676 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:56.093019 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216d037-5071-4b6a-8248-374e676f6c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:56.093735 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216d037-5071-4b6a-8248-374e676f6c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:56.094207 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216d037-5071-4b6a-8248-374e676f6c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:56.094562 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216d037-5071-4b6a-8248-374e676f6c90 admin admin] PATH_INFO: `/v3/users/bc56244c98bd4dbd896cb7f3911b4502` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:56.096206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216d037-5071-4b6a-8248-374e676f6c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc56244c98bd4dbd896cb7f3911b4502)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:56.099004 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216d037-5071-4b6a-8248-374e676f6c90 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:56.168411 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9216d037-5071-4b6a-8248-374e676f6c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc56244c98bd4dbd896cb7f3911b4502'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.170877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '881fc8c528cd40a3b5dbf4c49b467b14', 'request_id': 'req-efb84db2-7742-4818-a2ff-90bb345cb115'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.178180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '881fc8c528cd40a3b5dbf4c49b467b14', 'request_id': 'req-efb84db2-7742-4818-a2ff-90bb345cb115'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.186859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '881fc8c528cd40a3b5dbf4c49b467b14', 'request_id': 'req-efb84db2-7742-4818-a2ff-90bb345cb115'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.189572 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9216d037-5071-4b6a-8248-374e676f6c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc56244c98bd4dbd896cb7f3911b4502'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.190865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9216d037-5071-4b6a-8248-374e676f6c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc56244c98bd4dbd896cb7f3911b4502'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.192314 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9216d037-5071-4b6a-8248-374e676f6c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc56244c98bd4dbd896cb7f3911b4502'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.204234 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9216d037-5071-4b6a-8248-374e676f6c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc56244c98bd4dbd896cb7f3911b4502'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.213566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] The token cache is being invalidate because project 881fc8c528cd40a3b5dbf4c49b467b14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:56.213915 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.221400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.222453 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1405/2826] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:56 2022] DELETE /identity/v3/projects/881fc8c528cd40a3b5dbf4c49b467b14 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:56.225718 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:56.227406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:56.239807 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1422/2827] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:56 2022] DELETE /identity/v3/users/bc56244c98bd4dbd896cb7f3911b4502 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:56.244285 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f0cfa-11d0-462c-9e8e-4b8e06d18b14 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:56.244779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f0cfa-11d0-462c-9e8e-4b8e06d18b14 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:56.245122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f0cfa-11d0-462c-9e8e-4b8e06d18b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:56.250292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:56.273940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c88f0cfa-11d0-462c-9e8e-4b8e06d18b14 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:56.275159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c88f0cfa-11d0-462c-9e8e-4b8e06d18b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:56.291993 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:56.346789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb84db2-7742-4818-a2ff-90bb345cb115 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:56.372057 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1423/2828] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:56.384879 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f0cfa-11d0-462c-9e8e-4b8e06d18b14 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:56.386723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c88f0cfa-11d0-462c-9e8e-4b8e06d18b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:56.453761 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:56.454674 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:56.455106 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:56.455558 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] PATH_INFO: `/v3/users/da7cdf3b780744a997811b881ece0b9c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:56.461163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da7cdf3b780744a997811b881ece0b9c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:56.463038 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:56.490584 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f0cfa-11d0-462c-9e8e-4b8e06d18b14 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:56.495307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01742254-48bf-4321-aa32-59e782c64c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:56.496097 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01742254-48bf-4321-aa32-59e782c64c6e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:56.496456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01742254-48bf-4321-aa32-59e782c64c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:56.496865 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01742254-48bf-4321-aa32-59e782c64c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:56.516995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01742254-48bf-4321-aa32-59e782c64c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:56.520521 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01742254-48bf-4321-aa32-59e782c64c6e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:56.533116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1424/2829] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:55:56.548150 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da7cdf3b780744a997811b881ece0b9c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.549146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da7cdf3b780744a997811b881ece0b9c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.561974 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da7cdf3b780744a997811b881ece0b9c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.574267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da7cdf3b780744a997811b881ece0b9c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.575478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da7cdf3b780744a997811b881ece0b9c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:56.599412 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1406/2830] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:56 2022] DELETE /identity/v3/users/da7cdf3b780744a997811b881ece0b9c => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:57.058949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01742254-48bf-4321-aa32-59e782c64c6e service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:57.060679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-01742254-48bf-4321-aa32-59e782c64c6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:57.078110 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01742254-48bf-4321-aa32-59e782c64c6e service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:57.146212 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:57.148987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:57.198005 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:57.199273 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:57.199792 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:57.200310 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] PATH_INFO: `/v3/projects/7ea465e2cddc4badb834b3fd45ea14cc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:57.201832 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea465e2cddc4badb834b3fd45ea14cc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:57.204169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:57.206171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:57.243089 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c36cf-ec84-4f30-9c92-d17b49a5d937 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:57.248283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:57.248681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:57.248963 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:57.249282 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] PATH_INFO: `/v3/projects/f32a1cd0a01e40d1a42cbf3d780f3012` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:57.251519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f32a1cd0a01e40d1a42cbf3d780f3012)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:57.253859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:57.290800 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea465e2cddc4badb834b3fd45ea14cc', 'request_id': 'req-49594af6-24ca-4bd8-8e64-d298537ab547'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.299721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea465e2cddc4badb834b3fd45ea14cc', 'request_id': 'req-49594af6-24ca-4bd8-8e64-d298537ab547'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.307084 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:57.308930 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea465e2cddc4badb834b3fd45ea14cc', 'request_id': 'req-49594af6-24ca-4bd8-8e64-d298537ab547'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.323981 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f32a1cd0a01e40d1a42cbf3d780f3012', 'request_id': 'req-7de935c6-3dec-4384-8439-808aa9b6fc07'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.327108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] The token cache is being invalidate because project 7ea465e2cddc4badb834b3fd45ea14cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:57.327732 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.328750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.330428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1425/2831] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:57 2022] DELETE /identity/v3/projects/7ea465e2cddc4badb834b3fd45ea14cc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:57.331298 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f32a1cd0a01e40d1a42cbf3d780f3012', 'request_id': 'req-7de935c6-3dec-4384-8439-808aa9b6fc07'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.334720 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:57.336217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:57.338384 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f32a1cd0a01e40d1a42cbf3d780f3012', 'request_id': 'req-7de935c6-3dec-4384-8439-808aa9b6fc07'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.357617 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] The token cache is being invalidate because project f32a1cd0a01e40d1a42cbf3d780f3012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:57.357940 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.358710 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.360543 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1407/2832] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:57 2022] DELETE /identity/v3/projects/f32a1cd0a01e40d1a42cbf3d780f3012 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:55:57.365518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:57.370039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:57.430117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49594af6-24ca-4bd8-8e64-d298537ab547 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:57.507063 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de935c6-3dec-4384-8439-808aa9b6fc07 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:57.517454 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:57.518118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:57.518402 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:57.518681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] PATH_INFO: `/v3/users/248ae23b98114609bdd5701f27f49dde` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:57.523490 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:57.524352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=248ae23b98114609bdd5701f27f49dde)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:57.525274 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:57.525563 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:57.525840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] PATH_INFO: `/v3/users/81051acf9c2c4719911f2725e40e70e5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:57.528268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:57.529333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] RBAC: Authorizing `identity:delete_user(user_id=81051acf9c2c4719911f2725e40e70e5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:57.534050 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:57.617425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '248ae23b98114609bdd5701f27f49dde'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.618173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '248ae23b98114609bdd5701f27f49dde'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.630772 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '248ae23b98114609bdd5701f27f49dde'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.638880 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81051acf9c2c4719911f2725e40e70e5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.643850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '248ae23b98114609bdd5701f27f49dde'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.645997 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '248ae23b98114609bdd5701f27f49dde'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.652891 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:57.659522 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81051acf9c2c4719911f2725e40e70e5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.661435 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81051acf9c2c4719911f2725e40e70e5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.662109 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81051acf9c2c4719911f2725e40e70e5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.674195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1408/2833] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:57 2022] DELETE /identity/v3/users/248ae23b98114609bdd5701f27f49dde => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:57.693318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81051acf9c2c4719911f2725e40e70e5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:57.741535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1426/2834] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:55:57 2022] DELETE /identity/v3/users/81051acf9c2c4719911f2725e40e70e5 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:58.311560 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:58.312913 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:58.445998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:58.447103 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:58.475606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a06e9-d38d-4d55-af5a-207cb8a2f2f9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:58.485830 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:58.577843 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e643a-e836-43a7-b09d-d2f95e2ecedf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:58.582304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:58.583466 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:58.584159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:58.584678 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:58.584951 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:58.585187 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:58.585535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] PATH_INFO: `/v3/projects/0e8a1f33bbc74062b1232a8986386a6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:58.585657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] PATH_INFO: `/v3/projects/8c97413bfeec4aa589b380103c705b29` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:58.586910 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e8a1f33bbc74062b1232a8986386a6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:58.589258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:58.600833 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c97413bfeec4aa589b380103c705b29)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:58.604872 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:58.671066 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e8a1f33bbc74062b1232a8986386a6c', 'request_id': 'req-3b54289c-318d-4b99-8f29-4aacc19b8bbd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:58.684336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e8a1f33bbc74062b1232a8986386a6c', 'request_id': 'req-3b54289c-318d-4b99-8f29-4aacc19b8bbd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:58.707678 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e8a1f33bbc74062b1232a8986386a6c', 'request_id': 'req-3b54289c-318d-4b99-8f29-4aacc19b8bbd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:58.709168 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c97413bfeec4aa589b380103c705b29', 'request_id': 'req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:58.726800 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] The token cache is being invalidate because project 0e8a1f33bbc74062b1232a8986386a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:58.727281 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:58.728251 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3b54289c-318d-4b99-8f29-4aacc19b8bbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:58.730878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c97413bfeec4aa589b380103c705b29', 'request_id': 'req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:58.735431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1427/2835] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:58 2022] DELETE /identity/v3/projects/0e8a1f33bbc74062b1232a8986386a6c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:58.736460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafc158-5869-4844-aea6-20378e8b21a0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:58.736770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafc158-5869-4844-aea6-20378e8b21a0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:58.737535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafc158-5869-4844-aea6-20378e8b21a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:58.755497 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c97413bfeec4aa589b380103c705b29', 'request_id': 'req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:58.763690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3eafc158-5869-4844-aea6-20378e8b21a0 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:58.764941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3eafc158-5869-4844-aea6-20378e8b21a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:58.785996 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] The token cache is being invalidate because project 8c97413bfeec4aa589b380103c705b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:55:58.786506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:58.787394 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d5da6a1a-8ceb-49c3-b155-3ea1d523fbbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:55:58.789906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1409/2836] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:55:58 2022] DELETE /identity/v3/projects/8c97413bfeec4aa589b380103c705b29 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:58.900917 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1428/2837] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:58.911360 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc149940-eaeb-45e9-8704-10d1ef762252 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:58.911721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc149940-eaeb-45e9-8704-10d1ef762252 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:58.913195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc149940-eaeb-45e9-8704-10d1ef762252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:58.974423 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-fc149940-eaeb-45e9-8704-10d1ef762252 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:58.976107 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fc149940-eaeb-45e9-8704-10d1ef762252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:59.005034 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafc158-5869-4844-aea6-20378e8b21a0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.008233 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1410/2838] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:59.023493 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafc158-5869-4844-aea6-20378e8b21a0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.030535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9821bb3-307c-45d0-bf53-24ee4e655480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.031656 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9821bb3-307c-45d0-bf53-24ee4e655480 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.036388 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9821bb3-307c-45d0-bf53-24ee4e655480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.037438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9821bb3-307c-45d0-bf53-24ee4e655480 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.038873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9821bb3-307c-45d0-bf53-24ee4e655480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:55:59.039383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9821bb3-307c-45d0-bf53-24ee4e655480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:59.041863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9821bb3-307c-45d0-bf53-24ee4e655480 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:59.059159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1429/2839] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:55:59.063797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc149940-eaeb-45e9-8704-10d1ef762252 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.075519 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc149940-eaeb-45e9-8704-10d1ef762252 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.083486 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa019c2-f3f3-42ab-899f-076a5f7c4d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.084246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa019c2-f3f3-42ab-899f-076a5f7c4d2c admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.085563 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa019c2-f3f3-42ab-899f-076a5f7c4d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.085563 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa019c2-f3f3-42ab-899f-076a5f7c4d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.085563 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa019c2-f3f3-42ab-899f-076a5f7c4d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:59.088248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa019c2-f3f3-42ab-899f-076a5f7c4d2c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:59.108244 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:59.127667 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1411/2840] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:59 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:59.129337 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9821bb3-307c-45d0-bf53-24ee4e655480 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.140839 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9821bb3-307c-45d0-bf53-24ee4e655480 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.176724 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:59.209121 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829f2dc-f2f3-4152-bd5e-7ac19e7ed1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.209850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829f2dc-f2f3-4152-bd5e-7ac19e7ed1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.210140 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829f2dc-f2f3-4152-bd5e-7ac19e7ed1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.210613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829f2dc-f2f3-4152-bd5e-7ac19e7ed1e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.211416 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829f2dc-f2f3-4152-bd5e-7ac19e7ed1e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:59.213632 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829f2dc-f2f3-4152-bd5e-7ac19e7ed1e5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:59.241903 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1430/2841] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:59 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:59.244185 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa019c2-f3f3-42ab-899f-076a5f7c4d2c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.256834 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa019c2-f3f3-42ab-899f-076a5f7c4d2c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.263076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9771822-27b5-4b56-96e7-3ae07d4dc198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.263826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9771822-27b5-4b56-96e7-3ae07d4dc198 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.264243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9771822-27b5-4b56-96e7-3ae07d4dc198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.264750 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9771822-27b5-4b56-96e7-3ae07d4dc198 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.265479 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9771822-27b5-4b56-96e7-3ae07d4dc198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:59.267220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9771822-27b5-4b56-96e7-3ae07d4dc198 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:59.285757 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1412/2842] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:59.289517 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829f2dc-f2f3-4152-bd5e-7ac19e7ed1e5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.301947 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829f2dc-f2f3-4152-bd5e-7ac19e7ed1e5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.324188 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41781a8-4479-4500-946d-b83bad28afce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.342371 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41781a8-4479-4500-946d-b83bad28afce admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.343016 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41781a8-4479-4500-946d-b83bad28afce admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.343460 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41781a8-4479-4500-946d-b83bad28afce admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.344625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41781a8-4479-4500-946d-b83bad28afce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:59.346458 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41781a8-4479-4500-946d-b83bad28afce admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:59.359231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1431/2843] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:55:59.363867 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9771822-27b5-4b56-96e7-3ae07d4dc198 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.375076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9771822-27b5-4b56-96e7-3ae07d4dc198 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.380399 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b24be-dc5b-4bb4-b4d5-2dcc02207d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.381292 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b24be-dc5b-4bb4-b4d5-2dcc02207d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.381433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b24be-dc5b-4bb4-b4d5-2dcc02207d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.382310 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b24be-dc5b-4bb4-b4d5-2dcc02207d76 admin admin] PATH_INFO: `/v3/projects/5f6b7c87cdc94141bc2554ef488cc89a/users/2dd5307890474b328af31364959afa0b/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.410281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b24be-dc5b-4bb4-b4d5-2dcc02207d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f6b7c87cdc94141bc2554ef488cc89a, user_id=2dd5307890474b328af31364959afa0b, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:59.414518 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b24be-dc5b-4bb4-b4d5-2dcc02207d76 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:59.434095 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1413/2844] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:59 2022] PUT /identity/v3/projects/5f6b7c87cdc94141bc2554ef488cc89a/users/2dd5307890474b328af31364959afa0b/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:55:59.445590 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8c63b-a1b0-42ab-9ce4-fffb94c3a994 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.445876 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8c63b-a1b0-42ab-9ce4-fffb94c3a994 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.446193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8c63b-a1b0-42ab-9ce4-fffb94c3a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.483509 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-73e8c63b-a1b0-42ab-9ce4-fffb94c3a994 None None] MFA Rules not processed for user `2dd5307890474b328af31364959afa0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:59.484820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-73e8c63b-a1b0-42ab-9ce4-fffb94c3a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:59.579515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1432/2845] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:55:59.584905 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b24be-dc5b-4bb4-b4d5-2dcc02207d76 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.624565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d4010-542a-4ea9-ab60-708169e0a10f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.624872 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d4010-542a-4ea9-ab60-708169e0a10f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.625126 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d4010-542a-4ea9-ab60-708169e0a10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.656941 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8f5d4010-542a-4ea9-ab60-708169e0a10f None None] MFA Rules not processed for user `c6d4923a511342248e34954fc44811c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:55:59.658299 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8f5d4010-542a-4ea9-ab60-708169e0a10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:59.663763 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b24be-dc5b-4bb4-b4d5-2dcc02207d76 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.670896 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae8e4b-9e90-4c75-8410-90495e4a87a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.672024 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae8e4b-9e90-4c75-8410-90495e4a87a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.672316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae8e4b-9e90-4c75-8410-90495e4a87a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.672643 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae8e4b-9e90-4c75-8410-90495e4a87a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.673296 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae8e4b-9e90-4c75-8410-90495e4a87a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:59.689114 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae8e4b-9e90-4c75-8410-90495e4a87a6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:59.729071 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1414/2846] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:59 2022] POST /identity/v3/projects => generated 389 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:59.733430 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae8e4b-9e90-4c75-8410-90495e4a87a6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.746036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1433/2847] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:55:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:55:59.750267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d4010-542a-4ea9-ab60-708169e0a10f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.751661 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8f5d4010-542a-4ea9-ab60-708169e0a10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:55:59.767234 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d4010-542a-4ea9-ab60-708169e0a10f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.801929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:59.815575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae8e4b-9e90-4c75-8410-90495e4a87a6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.822157 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da81d6-0a14-41bc-b12e-0d94a7838f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.822781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da81d6-0a14-41bc-b12e-0d94a7838f4e admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.823070 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da81d6-0a14-41bc-b12e-0d94a7838f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.823345 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da81d6-0a14-41bc-b12e-0d94a7838f4e admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.823987 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da81d6-0a14-41bc-b12e-0d94a7838f4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:59.826005 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da81d6-0a14-41bc-b12e-0d94a7838f4e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:59.836072 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c715c5d-f5e6-490c-9fa6-a83d5e158de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.836882 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c715c5d-f5e6-490c-9fa6-a83d5e158de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.837312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c715c5d-f5e6-490c-9fa6-a83d5e158de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.837598 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c715c5d-f5e6-490c-9fa6-a83d5e158de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.838238 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c715c5d-f5e6-490c-9fa6-a83d5e158de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:59.840748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c715c5d-f5e6-490c-9fa6-a83d5e158de3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:59.851012 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1415/2848] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:59 2022] POST /identity/v3/users => generated 301 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:55:59.855118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da81d6-0a14-41bc-b12e-0d94a7838f4e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.866510 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da81d6-0a14-41bc-b12e-0d94a7838f4e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.872218 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91490a-f2c8-4cb7-96a8-22aaa1cbc456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.872965 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91490a-f2c8-4cb7-96a8-22aaa1cbc456 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.873251 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91490a-f2c8-4cb7-96a8-22aaa1cbc456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.873794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91490a-f2c8-4cb7-96a8-22aaa1cbc456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.874171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91490a-f2c8-4cb7-96a8-22aaa1cbc456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:59.875868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91490a-f2c8-4cb7-96a8-22aaa1cbc456 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:55:59.877639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1434/2849] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:55:59 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:55:59.882054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c715c5d-f5e6-490c-9fa6-a83d5e158de3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.888450 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1416/2850] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:55:59.892586 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91490a-f2c8-4cb7-96a8-22aaa1cbc456 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:55:59.904294 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91490a-f2c8-4cb7-96a8-22aaa1cbc456 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.934576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:55:59.974304 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c715c5d-f5e6-490c-9fa6-a83d5e158de3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:55:59.982135 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d29aa-1f66-43d2-9b14-11171881e324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.982811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d29aa-1f66-43d2-9b14-11171881e324 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.983094 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d29aa-1f66-43d2-9b14-11171881e324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.983373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d29aa-1f66-43d2-9b14-11171881e324 admin admin] PATH_INFO: `/v3/projects/c9c756c2c20e4745b08e3ad1d6a32aef/users/5807f9648fd8498d8dbfe9d02065d819/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.985820 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f93d23-0054-42da-9bdc-30b269289530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:55:59.987325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f93d23-0054-42da-9bdc-30b269289530 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:55:59.987325 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f93d23-0054-42da-9bdc-30b269289530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:55:59.988390 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f93d23-0054-42da-9bdc-30b269289530 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:55:59.989691 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f93d23-0054-42da-9bdc-30b269289530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:55:59.991760 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f93d23-0054-42da-9bdc-30b269289530 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:00.008758 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d29aa-1f66-43d2-9b14-11171881e324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c756c2c20e4745b08e3ad1d6a32aef, user_id=5807f9648fd8498d8dbfe9d02065d819, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:00.011614 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d29aa-1f66-43d2-9b14-11171881e324 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:00.024528 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1435/2851] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:55:59 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:56:00.028638 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f93d23-0054-42da-9bdc-30b269289530 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:00.033174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1417/2852] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:55:59 2022] PUT /identity/v3/projects/c9c756c2c20e4745b08e3ad1d6a32aef/users/5807f9648fd8498d8dbfe9d02065d819/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:56:00.034352 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d29aa-1f66-43d2-9b14-11171881e324 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:00.042269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f93d23-0054-42da-9bdc-30b269289530 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:00.120440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbbd50-ae46-4925-b9e1-6d17fdac3c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:00.120962 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbbd50-ae46-4925-b9e1-6d17fdac3c6b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:00.121277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbbd50-ae46-4925-b9e1-6d17fdac3c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:00.121543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbbd50-ae46-4925-b9e1-6d17fdac3c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:00.122318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbbd50-ae46-4925-b9e1-6d17fdac3c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:00.123981 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbbd50-ae46-4925-b9e1-6d17fdac3c6b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:00.124834 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d29aa-1f66-43d2-9b14-11171881e324 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:00.132954 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf4f33-551d-4dd0-a4cd-2d0934541c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:00.133576 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf4f33-551d-4dd0-a4cd-2d0934541c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:00.133850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf4f33-551d-4dd0-a4cd-2d0934541c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:00.134123 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf4f33-551d-4dd0-a4cd-2d0934541c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:00.134766 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf4f33-551d-4dd0-a4cd-2d0934541c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:00.138770 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1436/2853] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:56:00.138942 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf4f33-551d-4dd0-a4cd-2d0934541c80 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:00.140367 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbbd50-ae46-4925-b9e1-6d17fdac3c6b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:00.152875 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbbd50-ae46-4925-b9e1-6d17fdac3c6b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:00.158064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1418/2854] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:56:00.159303 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf4f33-551d-4dd0-a4cd-2d0934541c80 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:00.161558 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaed6df-1c59-4835-8cd2-9d484d0efd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:00.162228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaed6df-1c59-4835-8cd2-9d484d0efd73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:00.162499 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaed6df-1c59-4835-8cd2-9d484d0efd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:00.162757 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaed6df-1c59-4835-8cd2-9d484d0efd73 admin admin] PATH_INFO: `/v3/projects/772d2692c07943bfafc09b08345f8e23/users/d5b17c9bb91d42439af617619744acb9/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:00.172850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf4f33-551d-4dd0-a4cd-2d0934541c80 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:00.178462 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaed6df-1c59-4835-8cd2-9d484d0efd73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=772d2692c07943bfafc09b08345f8e23, user_id=d5b17c9bb91d42439af617619744acb9, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:00.178677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14ae0d-8781-47ea-8836-ff95071c2d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:00.179229 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14ae0d-8781-47ea-8836-ff95071c2d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:00.179578 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14ae0d-8781-47ea-8836-ff95071c2d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:00.179935 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14ae0d-8781-47ea-8836-ff95071c2d4b admin admin] PATH_INFO: `/v3/projects/c9c756c2c20e4745b08e3ad1d6a32aef/users/5807f9648fd8498d8dbfe9d02065d819/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:00.181219 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaed6df-1c59-4835-8cd2-9d484d0efd73 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:00.182129 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14ae0d-8781-47ea-8836-ff95071c2d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c756c2c20e4745b08e3ad1d6a32aef, user_id=5807f9648fd8498d8dbfe9d02065d819, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:00.185062 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14ae0d-8781-47ea-8836-ff95071c2d4b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:00.201245 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1437/2855] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:56:00 2022] PUT /identity/v3/projects/772d2692c07943bfafc09b08345f8e23/users/d5b17c9bb91d42439af617619744acb9/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:56:00.205266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaed6df-1c59-4835-8cd2-9d484d0efd73 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:00.205437 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1419/2856] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:56:00 2022] PUT /identity/v3/projects/c9c756c2c20e4745b08e3ad1d6a32aef/users/5807f9648fd8498d8dbfe9d02065d819/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:56:00.217524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30818e9-8709-4386-b094-b9a7df18c260 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:00.217811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30818e9-8709-4386-b094-b9a7df18c260 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:00.218254 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30818e9-8709-4386-b094-b9a7df18c260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:00.239198 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:00.257055 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b30818e9-8709-4386-b094-b9a7df18c260 None None] MFA Rules not processed for user `5807f9648fd8498d8dbfe9d02065d819`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:00.258606 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b30818e9-8709-4386-b094-b9a7df18c260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:00.288559 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaed6df-1c59-4835-8cd2-9d484d0efd73 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:00.334712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1420/2857] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:00 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:56:00.345361 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30818e9-8709-4386-b094-b9a7df18c260 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:00.346822 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b30818e9-8709-4386-b094-b9a7df18c260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:00.350161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bec3b3-c29c-4edb-a416-7c5a6f099211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:00.350771 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec3b3-c29c-4edb-a416-7c5a6f099211 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:00.351055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec3b3-c29c-4edb-a416-7c5a6f099211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:00.351333 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec3b3-c29c-4edb-a416-7c5a6f099211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:00.352155 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bec3b3-c29c-4edb-a416-7c5a6f099211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:00.353647 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bec3b3-c29c-4edb-a416-7c5a6f099211 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:00.364465 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1438/2858] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:56:00.365762 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:00.368783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bec3b3-c29c-4edb-a416-7c5a6f099211 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:00.380252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bec3b3-c29c-4edb-a416-7c5a6f099211 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:00.385117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901df19a-34f9-4bf3-881c-b23c97190ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:00.385776 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901df19a-34f9-4bf3-881c-b23c97190ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:00.386061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901df19a-34f9-4bf3-881c-b23c97190ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:00.386340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901df19a-34f9-4bf3-881c-b23c97190ad7 admin admin] PATH_INFO: `/v3/projects/772d2692c07943bfafc09b08345f8e23/users/d5b17c9bb91d42439af617619744acb9/roles/6cb25aac3b3e48f888f9a84dcef654e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:00.388237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901df19a-34f9-4bf3-881c-b23c97190ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=772d2692c07943bfafc09b08345f8e23, user_id=d5b17c9bb91d42439af617619744acb9, role_id=6cb25aac3b3e48f888f9a84dcef654e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:00.390925 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901df19a-34f9-4bf3-881c-b23c97190ad7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:00.402336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1439/2859] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:56:00 2022] PUT /identity/v3/projects/772d2692c07943bfafc09b08345f8e23/users/d5b17c9bb91d42439af617619744acb9/roles/6cb25aac3b3e48f888f9a84dcef654e6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:00.410318 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30818e9-8709-4386-b094-b9a7df18c260 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:00.464156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b55b9-b02c-4095-9720-84cb35a1827a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:00.464807 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b55b9-b02c-4095-9720-84cb35a1827a service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:00.465096 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b55b9-b02c-4095-9720-84cb35a1827a service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:00.465375 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b55b9-b02c-4095-9720-84cb35a1827a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:00.532594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b55b9-b02c-4095-9720-84cb35a1827a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:00.534051 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b55b9-b02c-4095-9720-84cb35a1827a service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:00.545064 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1421/2860] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:00 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:56:00.705670 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901df19a-34f9-4bf3-881c-b23c97190ad7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:00.787335 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901df19a-34f9-4bf3-881c-b23c97190ad7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:00.792874 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:00.793548 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:00.793784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:00.794060 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] PATH_INFO: `/v3/users/2dd5307890474b328af31364959afa0b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:00.795205 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd5307890474b328af31364959afa0b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:00.797203 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:00.859080 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd5307890474b328af31364959afa0b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:00.861447 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:00.867436 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd5307890474b328af31364959afa0b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:00.868498 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd5307890474b328af31364959afa0b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:00.869334 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd5307890474b328af31364959afa0b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:00.879623 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd5307890474b328af31364959afa0b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:00.899853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1440/2861] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:56:00 2022] DELETE /identity/v3/users/2dd5307890474b328af31364959afa0b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:56:00.904462 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7d5ee-dd22-4a07-8410-c06df90aeb60 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:00.904797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7d5ee-dd22-4a07-8410-c06df90aeb60 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:00.905081 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7d5ee-dd22-4a07-8410-c06df90aeb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:00.926494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-bbc7d5ee-dd22-4a07-8410-c06df90aeb60 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:00.927730 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bbc7d5ee-dd22-4a07-8410-c06df90aeb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:01.001316 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1422/2862] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:01.013431 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:01.015275 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:01.082813 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91ce84-e516-46c0-bb4c-85a53c1086e7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:01.087454 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df185473-2e2a-425b-8dc6-799b38894d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:01.088230 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df185473-2e2a-425b-8dc6-799b38894d33 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:01.088681 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df185473-2e2a-425b-8dc6-799b38894d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:01.088826 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df185473-2e2a-425b-8dc6-799b38894d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:01.102723 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df185473-2e2a-425b-8dc6-799b38894d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:01.104129 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df185473-2e2a-425b-8dc6-799b38894d33 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:01.127516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1441/2863] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:01.559330 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7d5ee-dd22-4a07-8410-c06df90aeb60 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:01.561144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bbc7d5ee-dd22-4a07-8410-c06df90aeb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:01.575068 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7d5ee-dd22-4a07-8410-c06df90aeb60 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:01.579628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:01.580273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:01.580600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:01.580881 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] PATH_INFO: `/v3/projects/5f6b7c87cdc94141bc2554ef488cc89a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:01.582182 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f6b7c87cdc94141bc2554ef488cc89a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:01.584134 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:01.612797 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:01.663702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f6b7c87cdc94141bc2554ef488cc89a', 'request_id': 'req-0a24a095-1946-4893-8b07-88b53d3314d1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:01.671439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f6b7c87cdc94141bc2554ef488cc89a', 'request_id': 'req-0a24a095-1946-4893-8b07-88b53d3314d1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:01.690325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f6b7c87cdc94141bc2554ef488cc89a', 'request_id': 'req-0a24a095-1946-4893-8b07-88b53d3314d1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:01.704696 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] The token cache is being invalidate because project 5f6b7c87cdc94141bc2554ef488cc89a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:56:01.704994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:01.705658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:01.707461 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1423/2864] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:56:01 2022] DELETE /identity/v3/projects/5f6b7c87cdc94141bc2554ef488cc89a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:01.713278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df185473-2e2a-425b-8dc6-799b38894d33 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:01.730336 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df185473-2e2a-425b-8dc6-799b38894d33 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:01.812015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad6ebd-9b00-4f4e-b79e-b824aa9ca087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:01.813082 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad6ebd-9b00-4f4e-b79e-b824aa9ca087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:01.813466 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad6ebd-9b00-4f4e-b79e-b824aa9ca087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:01.813784 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad6ebd-9b00-4f4e-b79e-b824aa9ca087 admin admin] PATH_INFO: `/v3/users/5807f9648fd8498d8dbfe9d02065d819` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:01.815111 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad6ebd-9b00-4f4e-b79e-b824aa9ca087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5807f9648fd8498d8dbfe9d02065d819)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:01.818262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad6ebd-9b00-4f4e-b79e-b824aa9ca087 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:01.938892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15ad6ebd-9b00-4f4e-b79e-b824aa9ca087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5807f9648fd8498d8dbfe9d02065d819'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:01.952759 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15ad6ebd-9b00-4f4e-b79e-b824aa9ca087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5807f9648fd8498d8dbfe9d02065d819'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:01.954206 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15ad6ebd-9b00-4f4e-b79e-b824aa9ca087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5807f9648fd8498d8dbfe9d02065d819'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:01.955015 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15ad6ebd-9b00-4f4e-b79e-b824aa9ca087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5807f9648fd8498d8dbfe9d02065d819'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:01.968107 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15ad6ebd-9b00-4f4e-b79e-b824aa9ca087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5807f9648fd8498d8dbfe9d02065d819'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:01.984998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:02.026814 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1442/2865] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:56:01 2022] DELETE /identity/v3/users/5807f9648fd8498d8dbfe9d02065d819 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:02.418464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:02.421347 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:02.445933 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24a095-1946-4893-8b07-88b53d3314d1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:02.517769 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:02.518563 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:02.519615 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:02.519906 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] PATH_INFO: `/v3/projects/c9c756c2c20e4745b08e3ad1d6a32aef` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:02.521849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c756c2c20e4745b08e3ad1d6a32aef)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:02.523211 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:02.590711 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c756c2c20e4745b08e3ad1d6a32aef', 'request_id': 'req-1f23d739-5fc4-457f-b7ef-59866f817bb3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:02.598889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c756c2c20e4745b08e3ad1d6a32aef', 'request_id': 'req-1f23d739-5fc4-457f-b7ef-59866f817bb3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:02.605702 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c756c2c20e4745b08e3ad1d6a32aef', 'request_id': 'req-1f23d739-5fc4-457f-b7ef-59866f817bb3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:02.622372 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] The token cache is being invalidate because project c9c756c2c20e4745b08e3ad1d6a32aef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:56:02.622724 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:02.624595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1f23d739-5fc4-457f-b7ef-59866f817bb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:02.626425 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1424/2866] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:56:02 2022] DELETE /identity/v3/projects/c9c756c2c20e4745b08e3ad1d6a32aef => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:02.632355 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841629c-889e-4377-b44a-623416986098 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:02.632812 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841629c-889e-4377-b44a-623416986098 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:02.633279 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841629c-889e-4377-b44a-623416986098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:02.659268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1841629c-889e-4377-b44a-623416986098 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:02.661223 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1841629c-889e-4377-b44a-623416986098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:02.752520 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1443/2867] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:02.812449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda104c-6f48-4ed4-95d3-49b14105567d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:02.812449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda104c-6f48-4ed4-95d3-49b14105567d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:02.812449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda104c-6f48-4ed4-95d3-49b14105567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:02.835246 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cfda104c-6f48-4ed4-95d3-49b14105567d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:02.836481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cfda104c-6f48-4ed4-95d3-49b14105567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:02.848878 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1425/2868] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:02.850539 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841629c-889e-4377-b44a-623416986098 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:02.861086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841629c-889e-4377-b44a-623416986098 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:02.867225 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4bcb3-1449-44c6-a34e-b39925036e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:02.868026 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4bcb3-1449-44c6-a34e-b39925036e13 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:02.868156 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4bcb3-1449-44c6-a34e-b39925036e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:02.868384 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4bcb3-1449-44c6-a34e-b39925036e13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:02.869075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4bcb3-1449-44c6-a34e-b39925036e13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:56:02.870192 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4bcb3-1449-44c6-a34e-b39925036e13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:02.870974 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4bcb3-1449-44c6-a34e-b39925036e13 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:02.887626 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1444/2869] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:56:02.888924 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda104c-6f48-4ed4-95d3-49b14105567d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:02.898914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda104c-6f48-4ed4-95d3-49b14105567d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:02.904013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424eafa2-f320-4fbf-b91b-c9fd0023bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:02.905171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424eafa2-f320-4fbf-b91b-c9fd0023bab2 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:02.905171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424eafa2-f320-4fbf-b91b-c9fd0023bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:02.905353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424eafa2-f320-4fbf-b91b-c9fd0023bab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:02.905926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424eafa2-f320-4fbf-b91b-c9fd0023bab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:02.910571 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424eafa2-f320-4fbf-b91b-c9fd0023bab2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:02.963325 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1426/2870] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:56:02 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:56:02.967685 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4bcb3-1449-44c6-a34e-b39925036e13 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:02.984294 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4bcb3-1449-44c6-a34e-b39925036e13 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:03.068797 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd53f05-6ed2-4357-9c15-bfed14b06989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:03.070307 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd53f05-6ed2-4357-9c15-bfed14b06989 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:03.070608 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd53f05-6ed2-4357-9c15-bfed14b06989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:03.071352 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd53f05-6ed2-4357-9c15-bfed14b06989 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:03.072840 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd53f05-6ed2-4357-9c15-bfed14b06989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:03.078612 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd53f05-6ed2-4357-9c15-bfed14b06989 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:03.106571 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1445/2871] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:56:02 2022] POST /identity/v3/users => generated 286 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:56:03.107837 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424eafa2-f320-4fbf-b91b-c9fd0023bab2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:03.119584 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424eafa2-f320-4fbf-b91b-c9fd0023bab2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:03.124671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba3a55-8553-4809-85c8-ee4a536eb9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:03.125306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba3a55-8553-4809-85c8-ee4a536eb9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:03.125578 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba3a55-8553-4809-85c8-ee4a536eb9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:03.125836 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba3a55-8553-4809-85c8-ee4a536eb9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:03.126494 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba3a55-8553-4809-85c8-ee4a536eb9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:03.130782 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba3a55-8553-4809-85c8-ee4a536eb9e0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:03.132264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:03.147262 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1427/2872] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:56:03.154970 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd53f05-6ed2-4357-9c15-bfed14b06989 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:03.173990 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd53f05-6ed2-4357-9c15-bfed14b06989 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:03.187502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06f067-bdb3-4c01-b50c-d559c0455137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:03.192774 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06f067-bdb3-4c01-b50c-d559c0455137 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:03.193116 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06f067-bdb3-4c01-b50c-d559c0455137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:03.193397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06f067-bdb3-4c01-b50c-d559c0455137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:03.194085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06f067-bdb3-4c01-b50c-d559c0455137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:03.196515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06f067-bdb3-4c01-b50c-d559c0455137 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:03.233654 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1446/2873] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:56:03.237683 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba3a55-8553-4809-85c8-ee4a536eb9e0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:03.252235 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba3a55-8553-4809-85c8-ee4a536eb9e0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:03.259034 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce61a2-70a5-4663-b70a-10879441f1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:03.259717 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce61a2-70a5-4663-b70a-10879441f1a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:03.260000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce61a2-70a5-4663-b70a-10879441f1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:03.260277 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce61a2-70a5-4663-b70a-10879441f1a9 admin admin] PATH_INFO: `/v3/projects/7d245c5d92594d68b044d8df60c55baa/users/8c827d6608954cc695b92b0bfcca425d/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:03.279733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce61a2-70a5-4663-b70a-10879441f1a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d245c5d92594d68b044d8df60c55baa, user_id=8c827d6608954cc695b92b0bfcca425d, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:03.283434 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce61a2-70a5-4663-b70a-10879441f1a9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:03.322059 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1428/2874] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:56:03 2022] PUT /identity/v3/projects/7d245c5d92594d68b044d8df60c55baa/users/8c827d6608954cc695b92b0bfcca425d/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:03.324278 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599080a-cefd-4f3f-8720-538e7153d39a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:03.324811 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599080a-cefd-4f3f-8720-538e7153d39a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:03.325128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599080a-cefd-4f3f-8720-538e7153d39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:03.382003 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0599080a-cefd-4f3f-8720-538e7153d39a None None] MFA Rules not processed for user `8c827d6608954cc695b92b0bfcca425d`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:03.383266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0599080a-cefd-4f3f-8720-538e7153d39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:03.453242 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:03.490485 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1447/2875] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:03.496446 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce61a2-70a5-4663-b70a-10879441f1a9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:03.499169 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-39ce61a2-70a5-4663-b70a-10879441f1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:03.588562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce61a2-70a5-4663-b70a-10879441f1a9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:03.593145 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa6d68-3340-459d-bdc7-a12ce417cb67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:03.594555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa6d68-3340-459d-bdc7-a12ce417cb67 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:03.595093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa6d68-3340-459d-bdc7-a12ce417cb67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:03.595594 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa6d68-3340-459d-bdc7-a12ce417cb67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:03.606349 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa6d68-3340-459d-bdc7-a12ce417cb67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:03.608304 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa6d68-3340-459d-bdc7-a12ce417cb67 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:03.620128 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1429/2876] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:03.645119 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599080a-cefd-4f3f-8720-538e7153d39a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:03.730249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599080a-cefd-4f3f-8720-538e7153d39a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:03.735944 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:03.736762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:03.737067 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:03.737981 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] PATH_INFO: `/v3/users/8c827d6608954cc695b92b0bfcca425d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:03.739777 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c827d6608954cc695b92b0bfcca425d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:03.742169 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:03.858553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c827d6608954cc695b92b0bfcca425d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:03.871474 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c827d6608954cc695b92b0bfcca425d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:03.873636 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c827d6608954cc695b92b0bfcca425d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:03.874267 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c827d6608954cc695b92b0bfcca425d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:03.890250 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c827d6608954cc695b92b0bfcca425d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:03.927902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1448/2877] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:56:03 2022] DELETE /identity/v3/users/8c827d6608954cc695b92b0bfcca425d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:03.932566 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5d607-38f5-4021-ab1e-da8f3346dfc7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:03.933045 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5d607-38f5-4021-ab1e-da8f3346dfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:03.933520 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5d607-38f5-4021-ab1e-da8f3346dfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:04.007978 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-49b5d607-38f5-4021-ab1e-da8f3346dfc7 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:04.009526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-49b5d607-38f5-4021-ab1e-da8f3346dfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:04.161396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1430/2878] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:04.167228 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:04.169165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:04.314040 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d228b51-29a3-4f2b-92e2-336b9fd68094 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:04.326846 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0671e94-a10a-464b-b754-edbb66bed73f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:04.327493 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0671e94-a10a-464b-b754-edbb66bed73f service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:04.327891 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0671e94-a10a-464b-b754-edbb66bed73f service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:04.328083 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0671e94-a10a-464b-b754-edbb66bed73f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:04.364415 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0671e94-a10a-464b-b754-edbb66bed73f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:04.366214 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0671e94-a10a-464b-b754-edbb66bed73f service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:04.392424 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1449/2879] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:04.965280 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5d607-38f5-4021-ab1e-da8f3346dfc7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:04.966914 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-49b5d607-38f5-4021-ab1e-da8f3346dfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:04.982433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5d607-38f5-4021-ab1e-da8f3346dfc7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:04.993590 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:04.994700 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:04.995159 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:04.995609 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] PATH_INFO: `/v3/projects/7d245c5d92594d68b044d8df60c55baa` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:04.998826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d245c5d92594d68b044d8df60c55baa)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:05.001789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:05.123507 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d245c5d92594d68b044d8df60c55baa', 'request_id': 'req-e111ba3c-0700-4a4e-b027-78eb77c3c86b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:05.152843 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d245c5d92594d68b044d8df60c55baa', 'request_id': 'req-e111ba3c-0700-4a4e-b027-78eb77c3c86b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:05.164887 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d245c5d92594d68b044d8df60c55baa', 'request_id': 'req-e111ba3c-0700-4a4e-b027-78eb77c3c86b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:05.179926 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:05.210629 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] The token cache is being invalidate because project 7d245c5d92594d68b044d8df60c55baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:56:05.211697 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:05.213524 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e111ba3c-0700-4a4e-b027-78eb77c3c86b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:05.219517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1431/2880] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:56:04 2022] DELETE /identity/v3/projects/7d245c5d92594d68b044d8df60c55baa => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:05.221401 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed24b-0f25-40a7-996a-86fa620220b1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:05.222711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed24b-0f25-40a7-996a-86fa620220b1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:05.222711 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed24b-0f25-40a7-996a-86fa620220b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:05.249989 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:05.257502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2d6ed24b-0f25-40a7-996a-86fa620220b1 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:05.266033 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2d6ed24b-0f25-40a7-996a-86fa620220b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:05.375566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1450/2881] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:05.384283 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba2496-103a-4c70-915e-edb41af74ee6 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:05.384682 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba2496-103a-4c70-915e-edb41af74ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:05.384826 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba2496-103a-4c70-915e-edb41af74ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:05.408108 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8aba2496-103a-4c70-915e-edb41af74ee6 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:05.409713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8aba2496-103a-4c70-915e-edb41af74ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:05.422243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1432/2882] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:05.426697 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ed24b-0f25-40a7-996a-86fa620220b1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:05.440179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ed24b-0f25-40a7-996a-86fa620220b1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:05.452939 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb5f2b-aace-4e2b-b005-c1ae01748ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:05.454600 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb5f2b-aace-4e2b-b005-c1ae01748ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:05.454886 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb5f2b-aace-4e2b-b005-c1ae01748ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:05.455163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb5f2b-aace-4e2b-b005-c1ae01748ec3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:05.456851 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb5f2b-aace-4e2b-b005-c1ae01748ec3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:56:05.457501 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb5f2b-aace-4e2b-b005-c1ae01748ec3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:05.459241 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb5f2b-aace-4e2b-b005-c1ae01748ec3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:05.473590 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1451/2883] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:56:05.480431 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba2496-103a-4c70-915e-edb41af74ee6 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:05.491422 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba2496-103a-4c70-915e-edb41af74ee6 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:05.496640 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b6f52-cce8-47da-8634-0e8b502500f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:05.497334 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b6f52-cce8-47da-8634-0e8b502500f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:05.497529 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b6f52-cce8-47da-8634-0e8b502500f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:05.497795 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b6f52-cce8-47da-8634-0e8b502500f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:05.498512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b6f52-cce8-47da-8634-0e8b502500f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:05.500665 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b6f52-cce8-47da-8634-0e8b502500f2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:05.542733 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1433/2884] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:56:05 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:56:05.545255 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb5f2b-aace-4e2b-b005-c1ae01748ec3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:05.579266 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb5f2b-aace-4e2b-b005-c1ae01748ec3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:05.738585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09945c91-e722-4428-bb47-554b5491194b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:05.739417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09945c91-e722-4428-bb47-554b5491194b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:05.739591 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09945c91-e722-4428-bb47-554b5491194b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:05.739772 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09945c91-e722-4428-bb47-554b5491194b admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:05.740689 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09945c91-e722-4428-bb47-554b5491194b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:05.742978 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09945c91-e722-4428-bb47-554b5491194b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:05.783808 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1452/2885] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:56:05 2022] POST /identity/v3/users => generated 285 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:56:05.787910 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b6f52-cce8-47da-8634-0e8b502500f2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:05.822130 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b6f52-cce8-47da-8634-0e8b502500f2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:05.847932 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eea78-1018-4223-82e1-0003405f99f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:05.849022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eea78-1018-4223-82e1-0003405f99f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:05.849514 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eea78-1018-4223-82e1-0003405f99f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:05.849994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eea78-1018-4223-82e1-0003405f99f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:05.851162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19eea78-1018-4223-82e1-0003405f99f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:05.857503 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19eea78-1018-4223-82e1-0003405f99f1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:05.882088 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1434/2886] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:56:05.886578 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09945c91-e722-4428-bb47-554b5491194b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:05.898370 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09945c91-e722-4428-bb47-554b5491194b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:05.903486 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afd8f2-d642-4d9a-8552-505ca439ce6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:05.904108 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afd8f2-d642-4d9a-8552-505ca439ce6f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:05.904393 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afd8f2-d642-4d9a-8552-505ca439ce6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:05.904702 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afd8f2-d642-4d9a-8552-505ca439ce6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:05.905359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afd8f2-d642-4d9a-8552-505ca439ce6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:05.908399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afd8f2-d642-4d9a-8552-505ca439ce6f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:05.933890 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1453/2887] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:56:05.938781 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eea78-1018-4223-82e1-0003405f99f1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:05.962985 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eea78-1018-4223-82e1-0003405f99f1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:05.982413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d705c4a-2cb0-471b-95ee-ec7cbb6b54ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:05.983765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705c4a-2cb0-471b-95ee-ec7cbb6b54ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:05.985248 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705c4a-2cb0-471b-95ee-ec7cbb6b54ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:05.986500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705c4a-2cb0-471b-95ee-ec7cbb6b54ed admin admin] PATH_INFO: `/v3/projects/c2bd7420debc4ff4bd828cd7900f2da6/users/8ca168d3801741db9f5766430dc973d0/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:06.024293 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d705c4a-2cb0-471b-95ee-ec7cbb6b54ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2bd7420debc4ff4bd828cd7900f2da6, user_id=8ca168d3801741db9f5766430dc973d0, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:06.028867 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d705c4a-2cb0-471b-95ee-ec7cbb6b54ed admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:06.069332 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1435/2888] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:56:05 2022] PUT /identity/v3/projects/c2bd7420debc4ff4bd828cd7900f2da6/users/8ca168d3801741db9f5766430dc973d0/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:06.081967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c97b-7bac-4767-9f1b-80cb00c6a15f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:06.082351 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c97b-7bac-4767-9f1b-80cb00c6a15f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:06.082529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c97b-7bac-4767-9f1b-80cb00c6a15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:06.152224 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9b43c97b-7bac-4767-9f1b-80cb00c6a15f None None] MFA Rules not processed for user `8ca168d3801741db9f5766430dc973d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:06.153728 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9b43c97b-7bac-4767-9f1b-80cb00c6a15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:06.194213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5637f50-81ca-4191-b1aa-c2a523d9ee22 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:06.194591 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5637f50-81ca-4191-b1aa-c2a523d9ee22 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:06.194939 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5637f50-81ca-4191-b1aa-c2a523d9ee22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:06.242098 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b5637f50-81ca-4191-b1aa-c2a523d9ee22 None None] MFA Rules not processed for user `d5b17c9bb91d42439af617619744acb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:06.243443 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b5637f50-81ca-4191-b1aa-c2a523d9ee22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:06.307210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1454/2889] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:06.322594 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43c97b-7bac-4767-9f1b-80cb00c6a15f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:06.324585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9b43c97b-7bac-4767-9f1b-80cb00c6a15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:06.328552 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:06.388655 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1436/2890] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:56:06.408789 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5637f50-81ca-4191-b1aa-c2a523d9ee22 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:06.410528 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b5637f50-81ca-4191-b1aa-c2a523d9ee22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:06.463258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43c97b-7bac-4767-9f1b-80cb00c6a15f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:06.467322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77a8bd-7a51-47f7-af0b-987e45a78495 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ec048788904bc58245e878a8644091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:06.468262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77a8bd-7a51-47f7-af0b-987e45a78495 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:06.468639 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77a8bd-7a51-47f7-af0b-987e45a78495 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:06.468983 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77a8bd-7a51-47f7-af0b-987e45a78495 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:06.481055 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77a8bd-7a51-47f7-af0b-987e45a78495 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:06.482786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77a8bd-7a51-47f7-af0b-987e45a78495 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:06.496231 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1455/2891] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:56:06.499938 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5637f50-81ca-4191-b1aa-c2a523d9ee22 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:06.509112 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d86b5-f2f7-48db-af5a-286a555912b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:06.509859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d86b5-f2f7-48db-af5a-286a555912b7 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:06.510245 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d86b5-f2f7-48db-af5a-286a555912b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:06.510687 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d86b5-f2f7-48db-af5a-286a555912b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:06.513080 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d37d86b5-f2f7-48db-af5a-286a555912b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:06.534815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:06.593401 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d86b5-f2f7-48db-af5a-286a555912b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:06.595022 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d86b5-f2f7-48db-af5a-286a555912b7 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:06.605729 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1437/2892] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:09.049036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77a8bd-7a51-47f7-af0b-987e45a78495 service glance] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:09.051237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dc77a8bd-7a51-47f7-af0b-987e45a78495 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:09.146024 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77a8bd-7a51-47f7-af0b-987e45a78495 service glance] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:09.153565 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fa03f-9e8b-4b0a-8e48-8cd2e501453b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:09.154316 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fa03f-9e8b-4b0a-8e48-8cd2e501453b service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:09.154684 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fa03f-9e8b-4b0a-8e48-8cd2e501453b service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:09.155099 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fa03f-9e8b-4b0a-8e48-8cd2e501453b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:09.158217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-940fa03f-9e8b-4b0a-8e48-8cd2e501453b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:09.181850 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fa03f-9e8b-4b0a-8e48-8cd2e501453b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:09.183534 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fa03f-9e8b-4b0a-8e48-8cd2e501453b service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:09.198404 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1456/2893] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:16.600215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=20 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:19.193088 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=66 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:37.867286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f789-ddec-4fad-9712-3c69a683e916 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:37.867818 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f789-ddec-4fad-9712-3c69a683e916 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:37.867944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f789-ddec-4fad-9712-3c69a683e916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:37.887066 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0ff7f789-ddec-4fad-9712-3c69a683e916 None None] MFA Rules not processed for user `c6d4923a511342248e34954fc44811c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:37.888273 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0ff7f789-ddec-4fad-9712-3c69a683e916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:37.895240 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1438/2894] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:38.215828 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fa03f-9e8b-4b0a-8e48-8cd2e501453b service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:38.217249 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-940fa03f-9e8b-4b0a-8e48-8cd2e501453b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:38.273379 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fa03f-9e8b-4b0a-8e48-8cd2e501453b service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:38.277159 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33547972-837c-41ac-a3e3-ce1f75bf591b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:38.277779 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547972-837c-41ac-a3e3-ce1f75bf591b service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:38.278053 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547972-837c-41ac-a3e3-ce1f75bf591b service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:38.278318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547972-837c-41ac-a3e3-ce1f75bf591b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:38.280006 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-33547972-837c-41ac-a3e3-ce1f75bf591b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:38.290172 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33547972-837c-41ac-a3e3-ce1f75bf591b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:38.291908 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33547972-837c-41ac-a3e3-ce1f75bf591b service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:38.301117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1457/2895] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:38.764787 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7f789-ddec-4fad-9712-3c69a683e916 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:38.766203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0ff7f789-ddec-4fad-9712-3c69a683e916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:38.784842 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7f789-ddec-4fad-9712-3c69a683e916 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:38.790226 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b450685-1d72-4823-b008-294fb5c5e7af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:38.790847 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b450685-1d72-4823-b008-294fb5c5e7af service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:38.791131 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b450685-1d72-4823-b008-294fb5c5e7af service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:38.791409 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b450685-1d72-4823-b008-294fb5c5e7af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:38.793223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8b450685-1d72-4823-b008-294fb5c5e7af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:38.806200 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b450685-1d72-4823-b008-294fb5c5e7af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:38.807912 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b450685-1d72-4823-b008-294fb5c5e7af service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:38.827651 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1439/2896] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:42.997529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33547972-837c-41ac-a3e3-ce1f75bf591b service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:43.022729 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33547972-837c-41ac-a3e3-ce1f75bf591b service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:43.029516 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac6c4e-8a4e-44c6-a933-cfca92545c00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:43.030160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac6c4e-8a4e-44c6-a933-cfca92545c00 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:43.030469 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac6c4e-8a4e-44c6-a933-cfca92545c00 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:43.030767 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac6c4e-8a4e-44c6-a933-cfca92545c00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:43.033073 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d0ac6c4e-8a4e-44c6-a933-cfca92545c00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:43.047545 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac6c4e-8a4e-44c6-a933-cfca92545c00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:43.050122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac6c4e-8a4e-44c6-a933-cfca92545c00 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:43.062128 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1458/2897] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:46.178237 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b450685-1d72-4823-b008-294fb5c5e7af service placement] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:46.240126 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b450685-1d72-4823-b008-294fb5c5e7af service placement] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:46.244575 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:46.245198 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:46.245481 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:46.245758 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] PATH_INFO: `/v3/users/8ca168d3801741db9f5766430dc973d0` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:46.246869 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca168d3801741db9f5766430dc973d0)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:46.248818 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:46.295919 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca168d3801741db9f5766430dc973d0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:46.296612 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca168d3801741db9f5766430dc973d0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:46.304160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca168d3801741db9f5766430dc973d0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:46.311531 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca168d3801741db9f5766430dc973d0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:46.312163 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca168d3801741db9f5766430dc973d0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:46.331173 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1440/2898] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:56:46 2022] DELETE /identity/v3/users/8ca168d3801741db9f5766430dc973d0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:46.335614 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f19813-e613-4e97-9507-fa31e32feb4d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:46.336101 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f19813-e613-4e97-9507-fa31e32feb4d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:46.336217 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f19813-e613-4e97-9507-fa31e32feb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:46.355210 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c3f19813-e613-4e97-9507-fa31e32feb4d None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:46.356400 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c3f19813-e613-4e97-9507-fa31e32feb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:46.416001 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1459/2899] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:46.426567 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:46.428160 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:46.487175 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aefa8d-0fd0-4fda-b7d2-e8fbf9826fe2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:46.490892 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2105e3-e4dc-4476-abb8-f521120bf693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:46.491438 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2105e3-e4dc-4476-abb8-f521120bf693 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:46.491721 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2105e3-e4dc-4476-abb8-f521120bf693 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:46.491999 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2105e3-e4dc-4476-abb8-f521120bf693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:46.500849 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2105e3-e4dc-4476-abb8-f521120bf693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:46.503369 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2105e3-e4dc-4476-abb8-f521120bf693 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:46.512734 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1441/2900] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:46.778967 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f19813-e613-4e97-9507-fa31e32feb4d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:46.782054 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c3f19813-e613-4e97-9507-fa31e32feb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:46.796540 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f19813-e613-4e97-9507-fa31e32feb4d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:46.802768 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:46.803410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:46.803690 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:46.803966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] PATH_INFO: `/v3/projects/c2bd7420debc4ff4bd828cd7900f2da6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:46.805218 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2bd7420debc4ff4bd828cd7900f2da6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:46.807357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:46.874726 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2bd7420debc4ff4bd828cd7900f2da6', 'request_id': 'req-ba7a79d5-ba36-4030-97ed-5e6af10e9761'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:46.882858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2bd7420debc4ff4bd828cd7900f2da6', 'request_id': 'req-ba7a79d5-ba36-4030-97ed-5e6af10e9761'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:46.890248 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2bd7420debc4ff4bd828cd7900f2da6', 'request_id': 'req-ba7a79d5-ba36-4030-97ed-5e6af10e9761'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:46.906940 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] The token cache is being invalidate because project c2bd7420debc4ff4bd828cd7900f2da6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:56:46.907283 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:46.908607 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba7a79d5-ba36-4030-97ed-5e6af10e9761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:46.910137 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1460/2901] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:56:46 2022] DELETE /identity/v3/projects/c2bd7420debc4ff4bd828cd7900f2da6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:46.915208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a39f5b-a192-4ebd-92d0-951c6767b865 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:46.915511 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a39f5b-a192-4ebd-92d0-951c6767b865 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:46.915811 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a39f5b-a192-4ebd-92d0-951c6767b865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:46.923581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:46.938333 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-10a39f5b-a192-4ebd-92d0-951c6767b865 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:46.940406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-10a39f5b-a192-4ebd-92d0-951c6767b865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:47.003289 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1442/2902] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:47.010987 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231e572-0e9f-44ac-8b8d-7bdc3a165e4b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.011277 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231e572-0e9f-44ac-8b8d-7bdc3a165e4b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.011554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231e572-0e9f-44ac-8b8d-7bdc3a165e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.033211 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b231e572-0e9f-44ac-8b8d-7bdc3a165e4b None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:47.034359 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b231e572-0e9f-44ac-8b8d-7bdc3a165e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:47.042428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1461/2903] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:47.046525 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a39f5b-a192-4ebd-92d0-951c6767b865 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:47.056569 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a39f5b-a192-4ebd-92d0-951c6767b865 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:47.061203 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97ca7c-f029-44f4-a49c-37c896050d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:47.061803 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97ca7c-f029-44f4-a49c-37c896050d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.062099 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97ca7c-f029-44f4-a49c-37c896050d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.062367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97ca7c-f029-44f4-a49c-37c896050d65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.063013 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97ca7c-f029-44f4-a49c-37c896050d65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:56:47.063320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97ca7c-f029-44f4-a49c-37c896050d65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:47.065449 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97ca7c-f029-44f4-a49c-37c896050d65 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:47.077044 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1443/2904] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Aug 31 11:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:56:47.081861 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231e572-0e9f-44ac-8b8d-7bdc3a165e4b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:47.091052 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231e572-0e9f-44ac-8b8d-7bdc3a165e4b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:47.098535 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17978ee4-8f9d-4f2c-8e5e-5d498a3df1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:47.099168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17978ee4-8f9d-4f2c-8e5e-5d498a3df1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.099449 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17978ee4-8f9d-4f2c-8e5e-5d498a3df1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.099719 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17978ee4-8f9d-4f2c-8e5e-5d498a3df1e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.100350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17978ee4-8f9d-4f2c-8e5e-5d498a3df1e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:47.102318 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17978ee4-8f9d-4f2c-8e5e-5d498a3df1e0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:47.106926 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:47.132972 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1462/2905] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Aug 31 11:56:47 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:56:47.137208 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97ca7c-f029-44f4-a49c-37c896050d65 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:47.148118 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97ca7c-f029-44f4-a49c-37c896050d65 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:47.208408 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece5264-5b60-48d5-81f1-041d232c5df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:47.210888 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece5264-5b60-48d5-81f1-041d232c5df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.211177 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece5264-5b60-48d5-81f1-041d232c5df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.211459 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece5264-5b60-48d5-81f1-041d232c5df0 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.212113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece5264-5b60-48d5-81f1-041d232c5df0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:47.214171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece5264-5b60-48d5-81f1-041d232c5df0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:47.236512 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1444/2906] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Aug 31 11:56:47 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:56:47.240604 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17978ee4-8f9d-4f2c-8e5e-5d498a3df1e0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:47.250737 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17978ee4-8f9d-4f2c-8e5e-5d498a3df1e0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:47.255679 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c114e41-884c-4fb7-a18a-8674f09b650f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:47.256268 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c114e41-884c-4fb7-a18a-8674f09b650f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.256585 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c114e41-884c-4fb7-a18a-8674f09b650f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.256860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c114e41-884c-4fb7-a18a-8674f09b650f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.257500 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c114e41-884c-4fb7-a18a-8674f09b650f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:47.259087 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c114e41-884c-4fb7-a18a-8674f09b650f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:47.270176 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1463/2907] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:56:47.274210 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece5264-5b60-48d5-81f1-041d232c5df0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:47.285227 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece5264-5b60-48d5-81f1-041d232c5df0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:47.289936 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876ad7-2ee8-4523-b7ed-e24b3f580e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:47.290535 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876ad7-2ee8-4523-b7ed-e24b3f580e7c admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.290816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876ad7-2ee8-4523-b7ed-e24b3f580e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.291093 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876ad7-2ee8-4523-b7ed-e24b3f580e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.291783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08876ad7-2ee8-4523-b7ed-e24b3f580e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:47.293600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08876ad7-2ee8-4523-b7ed-e24b3f580e7c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:47.304320 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1445/2908] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Aug 31 11:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:56:47.308417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c114e41-884c-4fb7-a18a-8674f09b650f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:47.318860 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c114e41-884c-4fb7-a18a-8674f09b650f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:47.323695 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a983610-8513-48f2-8f7b-615a3cb0dd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:47.324373 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a983610-8513-48f2-8f7b-615a3cb0dd7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.324688 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a983610-8513-48f2-8f7b-615a3cb0dd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.324966 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a983610-8513-48f2-8f7b-615a3cb0dd7b admin admin] PATH_INFO: `/v3/projects/37c30ec4be7c49db9853712317de2b07/users/88861306fc724fbb8ab7725856542687/roles/c70d64f54bff4d2ba00749c556be9c49` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.343154 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a983610-8513-48f2-8f7b-615a3cb0dd7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c30ec4be7c49db9853712317de2b07, user_id=88861306fc724fbb8ab7725856542687, role_id=c70d64f54bff4d2ba00749c556be9c49)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:47.346852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a983610-8513-48f2-8f7b-615a3cb0dd7b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:47.360118 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1464/2909] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Aug 31 11:56:47 2022] PUT /identity/v3/projects/37c30ec4be7c49db9853712317de2b07/users/88861306fc724fbb8ab7725856542687/roles/c70d64f54bff4d2ba00749c556be9c49 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:47.372225 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a595b-24b7-47ce-81cf-2575c5cb3785 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.372547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a595b-24b7-47ce-81cf-2575c5cb3785 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.372835 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a595b-24b7-47ce-81cf-2575c5cb3785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.402032 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6d3a595b-24b7-47ce-81cf-2575c5cb3785 None None] MFA Rules not processed for user `88861306fc724fbb8ab7725856542687`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:47.403252 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6d3a595b-24b7-47ce-81cf-2575c5cb3785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:47.464147 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:47.469945 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1446/2910] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:56:47.482036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a983610-8513-48f2-8f7b-615a3cb0dd7b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:47.483695 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5a983610-8513-48f2-8f7b-615a3cb0dd7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:47.543502 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a983610-8513-48f2-8f7b-615a3cb0dd7b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:47.547061 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26715d30-48ee-4e67-ae67-6a09a49739fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:47.547696 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26715d30-48ee-4e67-ae67-6a09a49739fb service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.548515 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26715d30-48ee-4e67-ae67-6a09a49739fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.548811 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26715d30-48ee-4e67-ae67-6a09a49739fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.557204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26715d30-48ee-4e67-ae67-6a09a49739fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:47.558659 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26715d30-48ee-4e67-ae67-6a09a49739fb service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:47.568131 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1465/2911] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:47.595162 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a595b-24b7-47ce-81cf-2575c5cb3785 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:47.658756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a595b-24b7-47ce-81cf-2575c5cb3785 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:47.663413 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:47.664036 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.664322 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.664633 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] PATH_INFO: `/v3/users/88861306fc724fbb8ab7725856542687` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.665773 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88861306fc724fbb8ab7725856542687)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:47.667692 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:47.717282 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88861306fc724fbb8ab7725856542687'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:47.717957 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88861306fc724fbb8ab7725856542687'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:47.724816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88861306fc724fbb8ab7725856542687'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:47.731928 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88861306fc724fbb8ab7725856542687'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:47.732604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88861306fc724fbb8ab7725856542687'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:47.750592 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1447/2912] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:56:47 2022] DELETE /identity/v3/users/88861306fc724fbb8ab7725856542687 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:47.755786 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b6207-86ad-45ea-bee8-72cbdfba3f99 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.756075 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b6207-86ad-45ea-bee8-72cbdfba3f99 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.756350 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b6207-86ad-45ea-bee8-72cbdfba3f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.775721 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a74b6207-86ad-45ea-bee8-72cbdfba3f99 None None] MFA Rules not processed for user `824ec611f32e40609c56e94f7edd552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:56:47.776949 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a74b6207-86ad-45ea-bee8-72cbdfba3f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:47.838511 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1466/2913] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:56:47.848350 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:47.849790 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:47.907952 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c8852-8907-4781-9730-a555114d3ce5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:47.911628 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf48e-373b-4a66-8201-5ad53a6bc490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:47.912230 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf48e-373b-4a66-8201-5ad53a6bc490 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:47.912574 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf48e-373b-4a66-8201-5ad53a6bc490 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:47.912850 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf48e-373b-4a66-8201-5ad53a6bc490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:47.921671 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbf48e-373b-4a66-8201-5ad53a6bc490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:47.924075 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbf48e-373b-4a66-8201-5ad53a6bc490 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:47.933406 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1448/2914] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:48.220215 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b6207-86ad-45ea-bee8-72cbdfba3f99 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:48.221683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a74b6207-86ad-45ea-bee8-72cbdfba3f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:48.235179 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b6207-86ad-45ea-bee8-72cbdfba3f99 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:48.241625 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:48.242262 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:48.242543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:48.242818 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] PATH_INFO: `/v3/projects/37c30ec4be7c49db9853712317de2b07` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:48.243929 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c30ec4be7c49db9853712317de2b07)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:48.245883 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:48.290844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:48.292357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c30ec4be7c49db9853712317de2b07', 'request_id': 'req-9ec37920-ad6b-418c-b732-13d580357c64'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:48.301428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c30ec4be7c49db9853712317de2b07', 'request_id': 'req-9ec37920-ad6b-418c-b732-13d580357c64'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:48.306998 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c30ec4be7c49db9853712317de2b07', 'request_id': 'req-9ec37920-ad6b-418c-b732-13d580357c64'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:48.319326 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] The token cache is being invalidate because project 37c30ec4be7c49db9853712317de2b07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:56:48.319618 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:48.320357 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:56:48.321897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1467/2915] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:56:48 2022] DELETE /identity/v3/projects/37c30ec4be7c49db9853712317de2b07 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:56:54.911619 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf48e-373b-4a66-8201-5ad53a6bc490 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:54.913045 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9fcbf48e-373b-4a66-8201-5ad53a6bc490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:54.936690 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:56:54.993270 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf48e-373b-4a66-8201-5ad53a6bc490 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:55.059650 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2bc1a-4c57-486e-a87e-5c842e28b695 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:55.060174 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2bc1a-4c57-486e-a87e-5c842e28b695 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:55.060456 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2bc1a-4c57-486e-a87e-5c842e28b695 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:55.060770 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2bc1a-4c57-486e-a87e-5c842e28b695 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:55.062526 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-62f2bc1a-4c57-486e-a87e-5c842e28b695 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:55.075752 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2bc1a-4c57-486e-a87e-5c842e28b695 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:55.077396 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2bc1a-4c57-486e-a87e-5c842e28b695 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:55.090367 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1449/2916] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:56:57.691425 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:56:57.692902 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:57.752932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec37920-ad6b-418c-b732-13d580357c64 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:56:57.756386 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ad93f-d5f6-4b1e-8a00-8c0cd81ee0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:56:57.757008 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ad93f-d5f6-4b1e-8a00-8c0cd81ee0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:56:57.757284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ad93f-d5f6-4b1e-8a00-8c0cd81ee0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:56:57.757553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ad93f-d5f6-4b1e-8a00-8c0cd81ee0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:56:57.759244 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3c0ad93f-d5f6-4b1e-8a00-8c0cd81ee0e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:56:57.821395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ad93f-d5f6-4b1e-8a00-8c0cd81ee0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:56:57.822853 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ad93f-d5f6-4b1e-8a00-8c0cd81ee0e5 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:56:57.832733 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1468/2917] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:56:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:57:05.085614 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=53 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:57:07.827980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=57,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:57:48.727464 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2bc1a-4c57-486e-a87e-5c842e28b695 service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:57:48.737133 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2bc1a-4c57-486e-a87e-5c842e28b695 service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:57:48.740760 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c85a2-1806-4e14-a04e-8ced9e148b7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:57:48.741353 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c85a2-1806-4e14-a04e-8ced9e148b7b service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:57:48.741636 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c85a2-1806-4e14-a04e-8ced9e148b7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:57:48.742025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c85a2-1806-4e14-a04e-8ced9e148b7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:57:48.743777 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-031c85a2-1806-4e14-a04e-8ced9e148b7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:57:48.753554 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c85a2-1806-4e14-a04e-8ced9e148b7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:57:48.755037 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c85a2-1806-4e14-a04e-8ced9e148b7b service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:57:48.762972 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1450/2918] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:57:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:57:48.895554 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61896a3-77b1-4dd8-8769-65342d7515cb None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:57:48.896163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61896a3-77b1-4dd8-8769-65342d7515cb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:57:48.896163 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61896a3-77b1-4dd8-8769-65342d7515cb None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:57:48.897476 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1469/2919] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:57:48.903323 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c1d59-e748-47ea-8249-a7c2486d2f3f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:57:48.903718 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c1d59-e748-47ea-8249-a7c2486d2f3f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:57:48.904085 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c1d59-e748-47ea-8249-a7c2486d2f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:57:48.943260 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e52c1d59-e748-47ea-8249-a7c2486d2f3f None None] MFA Rules not processed for user `30546ba7f5614f878a9ba322c7c55ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:57:48.944478 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e52c1d59-e748-47ea-8249-a7c2486d2f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:57:48.950986 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1451/2920] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:57:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:57:48.965543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61896a3-77b1-4dd8-8769-65342d7515cb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:57:48.967322 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d61896a3-77b1-4dd8-8769-65342d7515cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:57:48.981762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61896a3-77b1-4dd8-8769-65342d7515cb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:57:48.985677 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e5180-0104-47af-b903-8a2fdab3e1e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:57:48.986284 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5180-0104-47af-b903-8a2fdab3e1e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:57:48.986567 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5180-0104-47af-b903-8a2fdab3e1e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:57:48.986844 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5180-0104-47af-b903-8a2fdab3e1e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:57:48.994227 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e5180-0104-47af-b903-8a2fdab3e1e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:57:48.995750 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e5180-0104-47af-b903-8a2fdab3e1e2 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:57:49.005171 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1470/2921] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:57:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:57:49.096327 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20520b80-5346-4ba7-87ec-e474d148e2f2 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:57:49.096650 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20520b80-5346-4ba7-87ec-e474d148e2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:57:49.096931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20520b80-5346-4ba7-87ec-e474d148e2f2 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:57:49.098291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1452/2922] 10.222.0.60 () {54 vars in 921 bytes} [Wed Aug 31 11:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:57:49.103133 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933631b5-0d3b-4add-94ab-7ad04745fea1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:57:49.103426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933631b5-0d3b-4add-94ab-7ad04745fea1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:57:49.103738 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933631b5-0d3b-4add-94ab-7ad04745fea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:57:49.143134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-933631b5-0d3b-4add-94ab-7ad04745fea1 None None] MFA Rules not processed for user `f4048316ea784c389eb249b4dd915e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:57:49.144349 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-933631b5-0d3b-4add-94ab-7ad04745fea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:57:49.153617 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1471/2923] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Aug 31 11:57:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:57:49.164171 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20520b80-5346-4ba7-87ec-e474d148e2f2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:57:49.175785 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20520b80-5346-4ba7-87ec-e474d148e2f2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:57:49.179400 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379affcc-3a37-4fe4-b1c2-36d0767a470a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:57:49.179995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379affcc-3a37-4fe4-b1c2-36d0767a470a service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:57:49.180281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379affcc-3a37-4fe4-b1c2-36d0767a470a service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:57:49.180595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379affcc-3a37-4fe4-b1c2-36d0767a470a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:57:49.192113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379affcc-3a37-4fe4-b1c2-36d0767a470a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:57:49.193595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379affcc-3a37-4fe4-b1c2-36d0767a470a service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:57:49.204813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1453/2924] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:57:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:57:58.320291 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933631b5-0d3b-4add-94ab-7ad04745fea1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:57:58.331833 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933631b5-0d3b-4add-94ab-7ad04745fea1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:57:58.335537 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c31c5-4905-47c7-906a-bcd0347262c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:57:58.336139 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c31c5-4905-47c7-906a-bcd0347262c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:57:58.336417 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c31c5-4905-47c7-906a-bcd0347262c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:57:58.336762 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c31c5-4905-47c7-906a-bcd0347262c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:57:58.338482 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-835c31c5-4905-47c7-906a-bcd0347262c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:57:58.353869 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c31c5-4905-47c7-906a-bcd0347262c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:57:58.355290 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c31c5-4905-47c7-906a-bcd0347262c9 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:57:58.368775 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1472/2925] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:57:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:57:59.199637 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=21 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:57:59.555649 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379affcc-3a37-4fe4-b1c2-36d0767a470a service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:57:59.567324 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379affcc-3a37-4fe4-b1c2-36d0767a470a service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:57:59.570921 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46466c9e-8190-4818-8086-3b5fe1758d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:57:59.571517 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46466c9e-8190-4818-8086-3b5fe1758d14 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:57:59.571801 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46466c9e-8190-4818-8086-3b5fe1758d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:57:59.572077 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46466c9e-8190-4818-8086-3b5fe1758d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:57:59.573868 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-46466c9e-8190-4818-8086-3b5fe1758d14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:57:59.587865 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46466c9e-8190-4818-8086-3b5fe1758d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:57:59.589319 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46466c9e-8190-4818-8086-3b5fe1758d14 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:57:59.600213 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1454/2926] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:57:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:58:08.361486 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=21 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:58:09.595581 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:59:17.503269 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e91383-59d2-49b5-aa87-4e579622e577 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:59:17.503897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e91383-59d2-49b5-aa87-4e579622e577 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:59:17.503897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e91383-59d2-49b5-aa87-4e579622e577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:59:17.525923 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-77e91383-59d2-49b5-aa87-4e579622e577 None None] MFA Rules not processed for user `c6d4923a511342248e34954fc44811c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:59:17.527141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-77e91383-59d2-49b5-aa87-4e579622e577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:59:17.589529 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1473/2927] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 11:59:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:59:18.741220 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46466c9e-8190-4818-8086-3b5fe1758d14 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:59:18.751286 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46466c9e-8190-4818-8086-3b5fe1758d14 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:59:18.756474 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea17e1-c74f-4e40-8779-361ba731d07f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:59:18.757433 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea17e1-c74f-4e40-8779-361ba731d07f service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:59:18.757852 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea17e1-c74f-4e40-8779-361ba731d07f service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:59:18.758264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea17e1-c74f-4e40-8779-361ba731d07f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:59:18.760796 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0eea17e1-c74f-4e40-8779-361ba731d07f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:59:18.773017 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea17e1-c74f-4e40-8779-361ba731d07f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:59:18.774468 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea17e1-c74f-4e40-8779-361ba731d07f service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:59:18.784306 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1455/2928] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:59:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:59:26.763085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e91383-59d2-49b5-aa87-4e579622e577 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:59:26.773317 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e91383-59d2-49b5-aa87-4e579622e577 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:59:26.776866 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b519085-0469-442d-896f-b88a97bd2289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:59:26.777467 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519085-0469-442d-896f-b88a97bd2289 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:59:26.777753 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519085-0469-442d-896f-b88a97bd2289 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:59:26.778027 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519085-0469-442d-896f-b88a97bd2289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:59:26.779752 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8b519085-0469-442d-896f-b88a97bd2289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:59:26.850980 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b519085-0469-442d-896f-b88a97bd2289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:59:26.852432 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b519085-0469-442d-896f-b88a97bd2289 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:59:26.864823 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1474/2929] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:59:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:59:27.737215 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea17e1-c74f-4e40-8779-361ba731d07f service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:59:27.738794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0eea17e1-c74f-4e40-8779-361ba731d07f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:59:27.761598 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea17e1-c74f-4e40-8779-361ba731d07f service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:59:27.770611 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa4151-3e9a-4b2c-a811-4d678f40ec04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:59:27.772253 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa4151-3e9a-4b2c-a811-4d678f40ec04 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:59:27.772677 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa4151-3e9a-4b2c-a811-4d678f40ec04 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:59:27.773146 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa4151-3e9a-4b2c-a811-4d678f40ec04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:59:27.775500 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-23aa4151-3e9a-4b2c-a811-4d678f40ec04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:59:27.794641 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa4151-3e9a-4b2c-a811-4d678f40ec04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:59:27.796998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa4151-3e9a-4b2c-a811-4d678f40ec04 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:59:27.815035 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1456/2930] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:59:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:59:31.651031 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b519085-0469-442d-896f-b88a97bd2289 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:59:31.661892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b519085-0469-442d-896f-b88a97bd2289 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:59:31.665898 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bff86-e298-41f9-ac13-70758e7f1905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:59:31.666543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bff86-e298-41f9-ac13-70758e7f1905 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:59:31.666873 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bff86-e298-41f9-ac13-70758e7f1905 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:59:31.667195 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bff86-e298-41f9-ac13-70758e7f1905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:59:31.669177 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a48bff86-e298-41f9-ac13-70758e7f1905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:59:31.741541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48bff86-e298-41f9-ac13-70758e7f1905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:59:31.742985 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48bff86-e298-41f9-ac13-70758e7f1905 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:59:31.755783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1475/2931] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 11:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:59:37.808379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=12 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:59:40.150291 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa4151-3e9a-4b2c-a811-4d678f40ec04 service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:59:40.152860 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-23aa4151-3e9a-4b2c-a811-4d678f40ec04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:59:40.175688 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa4151-3e9a-4b2c-a811-4d678f40ec04 service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:59:40.180307 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:59:40.180982 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:59:40.181266 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:59:40.181547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] PATH_INFO: `/v3/users/b5cf28697ff04a3ebd70b780c10dd830` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:59:40.183104 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5cf28697ff04a3ebd70b780c10dd830)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:59:40.185084 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:59:40.245625 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5cf28697ff04a3ebd70b780c10dd830'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.246334 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5cf28697ff04a3ebd70b780c10dd830'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.255547 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5cf28697ff04a3ebd70b780c10dd830'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.263934 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5cf28697ff04a3ebd70b780c10dd830'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.265019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5cf28697ff04a3ebd70b780c10dd830'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.282528 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1457/2932] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:59:40 2022] DELETE /identity/v3/users/b5cf28697ff04a3ebd70b780c10dd830 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:59:40.463252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bff86-e298-41f9-ac13-70758e7f1905 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:59:40.471394 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:59:40.486402 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bff86-e298-41f9-ac13-70758e7f1905 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:59:40.543824 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:59:40.544373 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:59:40.544683 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:59:40.544957 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] PATH_INFO: `/v3/projects/2cfb480658dd40f1a1f0fd5c627e843c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:59:40.546748 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cfb480658dd40f1a1f0fd5c627e843c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:59:40.548694 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:59:40.595397 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cfb480658dd40f1a1f0fd5c627e843c', 'request_id': 'req-1a3fc11d-4d75-40b2-9d3f-a533bb374230'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.601648 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cfb480658dd40f1a1f0fd5c627e843c', 'request_id': 'req-1a3fc11d-4d75-40b2-9d3f-a533bb374230'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.607383 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cfb480658dd40f1a1f0fd5c627e843c', 'request_id': 'req-1a3fc11d-4d75-40b2-9d3f-a533bb374230'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.620396 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] The token cache is being invalidate because project 2cfb480658dd40f1a1f0fd5c627e843c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:59:40.620722 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.621340 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.622896 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1476/2933] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:59:40 2022] DELETE /identity/v3/projects/2cfb480658dd40f1a1f0fd5c627e843c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:59:40.627040 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:59:40.628691 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:59:40.642180 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9202023-7c84-4c0a-9bb5-fff6eeb235c1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:59:40.700816 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:59:40.701506 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:59:40.701794 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:59:40.702083 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] PATH_INFO: `/v3/users/ca01c59c7e70448fb7119347b2bb581d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:59:40.703388 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca01c59c7e70448fb7119347b2bb581d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:59:40.705595 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:59:40.754465 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca01c59c7e70448fb7119347b2bb581d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.755144 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca01c59c7e70448fb7119347b2bb581d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.761956 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca01c59c7e70448fb7119347b2bb581d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.769472 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca01c59c7e70448fb7119347b2bb581d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.770113 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca01c59c7e70448fb7119347b2bb581d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:40.777927 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:59:40.790109 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1458/2934] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 11:59:40 2022] DELETE /identity/v3/users/ca01c59c7e70448fb7119347b2bb581d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:59:40.969312 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:59:40.970945 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:59:40.988456 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fc11d-4d75-40b2-9d3f-a533bb374230 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:59:41.052086 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:59:41.052754 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:59:41.053036 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:59:41.053311 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] PATH_INFO: `/v3/projects/16d704b0ba214a3d9d55260ec5e13993` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:59:41.054541 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] RBAC: Authorizing `identity:delete_project(project_id=16d704b0ba214a3d9d55260ec5e13993)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:59:41.056595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:59:41.094459 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:59:41.104781 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16d704b0ba214a3d9d55260ec5e13993', 'request_id': 'req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:41.112085 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16d704b0ba214a3d9d55260ec5e13993', 'request_id': 'req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:41.124130 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16d704b0ba214a3d9d55260ec5e13993', 'request_id': 'req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:41.136876 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] The token cache is being invalidate because project 16d704b0ba214a3d9d55260ec5e13993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:59:41.137165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:41.137783 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:59:41.139852 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1477/2935] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 11:59:40 2022] DELETE /identity/v3/projects/16d704b0ba214a3d9d55260ec5e13993 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:59:50.790859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:59:51.133746 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:01:14.037802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:01:14.039309 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:01:14.184948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68379e-e056-4f4b-b211-b67977fa9f71 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:01:14.245222 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814bcda-8d13-4093-9d54-31793628229b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:01:14.245823 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814bcda-8d13-4093-9d54-31793628229b service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:01:14.246685 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814bcda-8d13-4093-9d54-31793628229b service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:01:14.247065 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814bcda-8d13-4093-9d54-31793628229b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:01:14.248859 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2814bcda-8d13-4093-9d54-31793628229b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:01:14.259944 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814bcda-8d13-4093-9d54-31793628229b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:01:14.261555 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814bcda-8d13-4093-9d54-31793628229b service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:01:14.272166 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1459/2936] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 12:01:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:01:24.267756 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:01:28.702566 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:01:28.703995 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:01:28.766858 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047d6da-c91b-4fa7-aa9f-4f836f26b08d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:01:28.770557 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba55da-c093-493d-9f94-f222a2e259ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8dec99551ff4c31a17247cde3515acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:01:28.771160 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba55da-c093-493d-9f94-f222a2e259ee service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:01:28.771440 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba55da-c093-493d-9f94-f222a2e259ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:01:28.771717 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba55da-c093-493d-9f94-f222a2e259ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:01:28.773512 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-69ba55da-c093-493d-9f94-f222a2e259ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:01:28.832798 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba55da-c093-493d-9f94-f222a2e259ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:01:28.834233 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba55da-c093-493d-9f94-f222a2e259ee service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:01:28.843258 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1478/2937] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 12:01:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:01:38.032356 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966c918-e341-4f85-a254-979de6cd1bf8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:01:38.032931 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966c918-e341-4f85-a254-979de6cd1bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:01:38.033048 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966c918-e341-4f85-a254-979de6cd1bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:01:38.057134 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a966c918-e341-4f85-a254-979de6cd1bf8 None None] MFA Rules not processed for user `c6d4923a511342248e34954fc44811c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:01:38.059439 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a966c918-e341-4f85-a254-979de6cd1bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:01:38.066736 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1460/2938] 10.222.0.60 () {58 vars in 958 bytes} [Wed Aug 31 12:01:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:01:38.839089 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=50 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:01:39.049161 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba55da-c093-493d-9f94-f222a2e259ee service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:01:39.050543 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-69ba55da-c093-493d-9f94-f222a2e259ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:01:39.065605 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba55da-c093-493d-9f94-f222a2e259ee service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:01:39.070756 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac012fb-cde4-4e22-a6db-738fd2e8e591 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:01:39.071347 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac012fb-cde4-4e22-a6db-738fd2e8e591 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:01:39.071628 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac012fb-cde4-4e22-a6db-738fd2e8e591 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:01:39.071900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac012fb-cde4-4e22-a6db-738fd2e8e591 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:01:39.073731 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cac012fb-cde4-4e22-a6db-738fd2e8e591 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:01:39.085122 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac012fb-cde4-4e22-a6db-738fd2e8e591 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:01:39.086680 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac012fb-cde4-4e22-a6db-738fd2e8e591 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:01:39.097428 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1479/2939] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 12:01:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:01:48.049006 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:01:49.092877 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:01:57.865768 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966c918-e341-4f85-a254-979de6cd1bf8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:01:57.867284 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a966c918-e341-4f85-a254-979de6cd1bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:01:57.958487 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966c918-e341-4f85-a254-979de6cd1bf8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:01:57.964091 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612acc42-ce07-4383-8a83-7855ebfa446f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4048316ea784c389eb249b4dd915e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:01:57.964720 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612acc42-ce07-4383-8a83-7855ebfa446f service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:01:57.965000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612acc42-ce07-4383-8a83-7855ebfa446f service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:01:57.965272 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612acc42-ce07-4383-8a83-7855ebfa446f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:01:57.967276 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-612acc42-ce07-4383-8a83-7855ebfa446f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:01:57.979648 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612acc42-ce07-4383-8a83-7855ebfa446f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:01:57.981268 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612acc42-ce07-4383-8a83-7855ebfa446f service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:01:57.991998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1461/2940] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 12:01:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:02:07.987382 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=41 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:01.190889 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac012fb-cde4-4e22-a6db-738fd2e8e591 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:01.191928 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cac012fb-cde4-4e22-a6db-738fd2e8e591 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:01.281916 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac012fb-cde4-4e22-a6db-738fd2e8e591 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:01.288984 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252da686-fc9e-4433-8b69-8ba610052e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:01.289613 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da686-fc9e-4433-8b69-8ba610052e7f service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:01.289892 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da686-fc9e-4433-8b69-8ba610052e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:01.290165 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da686-fc9e-4433-8b69-8ba610052e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:01.292399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-252da686-fc9e-4433-8b69-8ba610052e7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:01.323047 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252da686-fc9e-4433-8b69-8ba610052e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:01.324504 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252da686-fc9e-4433-8b69-8ba610052e7f service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:01.337181 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1480/2941] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 12:04:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:04:02.560019 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612acc42-ce07-4383-8a83-7855ebfa446f service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:02.561995 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-612acc42-ce07-4383-8a83-7855ebfa446f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:02.575195 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612acc42-ce07-4383-8a83-7855ebfa446f service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:02.579446 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fc5d2-bce3-42f9-a7e0-58f379813ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:02.580276 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fc5d2-bce3-42f9-a7e0-58f379813ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:02.580719 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fc5d2-bce3-42f9-a7e0-58f379813ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:02.581092 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fc5d2-bce3-42f9-a7e0-58f379813ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:02.583141 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-456fc5d2-bce3-42f9-a7e0-58f379813ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:02.601947 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fc5d2-bce3-42f9-a7e0-58f379813ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:02.603877 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fc5d2-bce3-42f9-a7e0-58f379813ba2 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:02.619618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1462/2942] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 12:04:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:04:11.332134 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=40 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:12.613604 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:14.097102 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252da686-fc9e-4433-8b69-8ba610052e7f service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:14.108212 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252da686-fc9e-4433-8b69-8ba610052e7f service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:14.112237 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ea953-0423-4d31-9c1d-7f0b534a38b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30546ba7f5614f878a9ba322c7c55ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:14.112864 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ea953-0423-4d31-9c1d-7f0b534a38b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:14.113141 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ea953-0423-4d31-9c1d-7f0b534a38b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:14.113410 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ea953-0423-4d31-9c1d-7f0b534a38b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:14.115168 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-471ea953-0423-4d31-9c1d-7f0b534a38b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:14.204389 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ea953-0423-4d31-9c1d-7f0b534a38b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:14.205897 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ea953-0423-4d31-9c1d-7f0b534a38b8 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:14.226399 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1481/2943] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 12:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:04:17.230562 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fc5d2-bce3-42f9-a7e0-58f379813ba2 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:17.232379 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-456fc5d2-bce3-42f9-a7e0-58f379813ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:17.255802 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fc5d2-bce3-42f9-a7e0-58f379813ba2 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:17.261305 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fad7-b442-45f0-bcb2-c1e9e6fd20c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:17.261973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fad7-b442-45f0-bcb2-c1e9e6fd20c5 service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:17.262264 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fad7-b442-45f0-bcb2-c1e9e6fd20c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:17.262556 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fad7-b442-45f0-bcb2-c1e9e6fd20c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:17.264813 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-17a6fad7-b442-45f0-bcb2-c1e9e6fd20c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:17.279858 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6fad7-b442-45f0-bcb2-c1e9e6fd20c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:17.281540 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6fad7-b442-45f0-bcb2-c1e9e6fd20c5 service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:17.293829 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1463/2944] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 12:04:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:04:20.575321 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ea953-0423-4d31-9c1d-7f0b534a38b8 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:20.601673 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ea953-0423-4d31-9c1d-7f0b534a38b8 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:20.607117 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cc0c0-e52f-4a2a-aa47-89bdb080d8bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:20.607900 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cc0c0-e52f-4a2a-aa47-89bdb080d8bd service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:20.608306 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cc0c0-e52f-4a2a-aa47-89bdb080d8bd service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:20.609662 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cc0c0-e52f-4a2a-aa47-89bdb080d8bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:20.613287 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c5cc0c0-e52f-4a2a-aa47-89bdb080d8bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:20.629878 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cc0c0-e52f-4a2a-aa47-89bdb080d8bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:20.632252 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cc0c0-e52f-4a2a-aa47-89bdb080d8bd service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:20.649245 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fad7-b442-45f0-bcb2-c1e9e6fd20c5 service placement] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:20.651889 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-17a6fad7-b442-45f0-bcb2-c1e9e6fd20c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:20.653553 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1482/2945] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 12:04:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:04:20.692103 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fad7-b442-45f0-bcb2-c1e9e6fd20c5 service placement] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:20.697589 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:20.698328 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:20.698765 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:20.699156 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] PATH_INFO: `/v3/users/c6d4923a511342248e34954fc44811c0` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:20.702618 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6d4923a511342248e34954fc44811c0)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:20.704658 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:20.756058 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6d4923a511342248e34954fc44811c0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:20.756779 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6d4923a511342248e34954fc44811c0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:20.764373 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6d4923a511342248e34954fc44811c0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:20.773039 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6d4923a511342248e34954fc44811c0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:20.773712 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6d4923a511342248e34954fc44811c0'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:20.791948 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1464/2946] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 12:04:20 2022] DELETE /identity/v3/users/c6d4923a511342248e34954fc44811c0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:20.978901 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cc0c0-e52f-4a2a-aa47-89bdb080d8bd service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:20.991766 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cc0c0-e52f-4a2a-aa47-89bdb080d8bd service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:21.059395 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:21.060138 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:21.060471 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:21.061051 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] PATH_INFO: `/v3/projects/fe46592aea54410c9f49a0371bb2bc73` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:21.062236 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe46592aea54410c9f49a0371bb2bc73)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:21.064438 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:21.115253 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe46592aea54410c9f49a0371bb2bc73', 'request_id': 'req-c1408061-c991-4109-9eb6-a8b24c09019b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.121633 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe46592aea54410c9f49a0371bb2bc73', 'request_id': 'req-c1408061-c991-4109-9eb6-a8b24c09019b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.122835 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:21.128653 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe46592aea54410c9f49a0371bb2bc73', 'request_id': 'req-c1408061-c991-4109-9eb6-a8b24c09019b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.143863 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] The token cache is being invalidate because project fe46592aea54410c9f49a0371bb2bc73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:04:21.144063 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.144742 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.147151 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1483/2947] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 12:04:20 2022] DELETE /identity/v3/projects/fe46592aea54410c9f49a0371bb2bc73 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:21.151243 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:21.152783 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:21.167492 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fd863-3fa6-4b22-be6a-99f7ac7a13f9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:21.227076 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:21.227716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:21.228000 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:21.228288 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] PATH_INFO: `/v3/users/d5b17c9bb91d42439af617619744acb9` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:21.229550 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5b17c9bb91d42439af617619744acb9)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:21.231600 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:21.285534 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5b17c9bb91d42439af617619744acb9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.286232 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5b17c9bb91d42439af617619744acb9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.293973 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5b17c9bb91d42439af617619744acb9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.297998 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:21.303281 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5b17c9bb91d42439af617619744acb9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.304223 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5b17c9bb91d42439af617619744acb9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.326457 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1465/2948] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Aug 31 12:04:21 2022] DELETE /identity/v3/users/d5b17c9bb91d42439af617619744acb9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:21.526022 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:21.528461 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:21.546270 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1408061-c991-4109-9eb6-a8b24c09019b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:21.609301 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824ec611f32e40609c56e94f7edd552e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927949f45e9a418783820746376979d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:21.610000 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:21.610204 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:21.610477 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] PATH_INFO: `/v3/projects/772d2692c07943bfafc09b08345f8e23` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:21.611595 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] RBAC: Authorizing `identity:delete_project(project_id=772d2692c07943bfafc09b08345f8e23)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:21.613642 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:21.666932 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '772d2692c07943bfafc09b08345f8e23', 'request_id': 'req-08153ca0-c189-4111-bab1-fa6e48144caa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.673847 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '772d2692c07943bfafc09b08345f8e23', 'request_id': 'req-08153ca0-c189-4111-bab1-fa6e48144caa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.685038 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '772d2692c07943bfafc09b08345f8e23', 'request_id': 'req-08153ca0-c189-4111-bab1-fa6e48144caa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.704193 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] The token cache is being invalidate because project 772d2692c07943bfafc09b08345f8e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:04:21.704426 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.705062 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-08153ca0-c189-4111-bab1-fa6e48144caa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:04:21.706546 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1484/2949] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Aug 31 12:04:21 2022] DELETE /identity/v3/projects/772d2692c07943bfafc09b08345f8e23 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:31.322739 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:31.700963 dv-cinder-839768-3-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=10,keystone:SELECT=55,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:17.004930 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:17.006368 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:17.083716 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780df836-dc7f-43f6-8326-527590f0365b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:17.153103 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa6227-6cbc-4144-a2bf-14976eb9058c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9114a71a3a247188b263d87b6cb12c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b4dae6d2b2345dc9259776a4237e4e6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:17.153713 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa6227-6cbc-4144-a2bf-14976eb9058c service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:17.153994 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa6227-6cbc-4144-a2bf-14976eb9058c service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:17.154267 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa6227-6cbc-4144-a2bf-14976eb9058c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:17.156025 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-64aa6227-6cbc-4144-a2bf-14976eb9058c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:17.168046 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa6227-6cbc-4144-a2bf-14976eb9058c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:17.169681 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa6227-6cbc-4144-a2bf-14976eb9058c service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:17.180657 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1466/2950] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Aug 31 12:06:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:06:27.175815 dv-cinder-839768-3-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=58 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}